Model-based Reinforcement Learning For Control Under Time-varying Dynamics

Abstract

Learning-based control methods typically assume stationary system dynamics, an assumption often violated in real-world systems due to drift, wear, or changing operating conditions. We study reinforcement learning for control under time-varying dynamics. We consider a continual model-based reinforcement learning setting in which an agent repeatedly learns and controls a dynamical system whose transition dynamics evolve across episodes. We analyze the problem using Gaussian process dynamics models under frequentist variation-budget assumptions. Our analysis shows that persistent non-stationarity requires explicitly limiting the influence of outdated data to maintain calibrated uncertainty and meaningful dynamic regret guarantees. Motivated by these insights, we propose a practical optimistic model-based reinforcement learning algorithm with adaptive data buffer mechanisms and demonstrate improved performance on continuous control benchmarks with non-stationary dynamics.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyiten2026model

Related papers

• Efficient Exploration In Continuous-time Model-based Reinforcement Learning (2023)0.00
• Demystifying Reinforcement Learning In Time-varying Systems (2022)0.00
• Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach (2023)0.00
• Continual Visual Reinforcement Learning With A Life-long World Model (2023)2.26
• An Optimal Policy For Learning Controllable Dynamics By Exploration (2025)0.00
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89
• Model-based Adaptation For Sample Efficient Transfer In Reinforcement Learning Control Of Parameter-varying Systems (2023)2.26
• Autoregressive Policies For Continuous Control Deep Reinforcement Learning (2019)7.50