Reinforcement Learning With Fast Stabilization In Linear Dynamical Systems

Abstract

In this work, we study model-based reinforcement learning (RL) in unknown stabilizable linear dynamical systems. When learning a dynamical system, one needs to stabilize the unknown dynamics in order to avoid system blow-ups. We propose an algorithm that certifies fast stabilization of the underlying system by effectively exploring the environment with an improved exploration strategy. We show that the proposed algorithm attains \(\tilde\{\mathcal\{O\}\}(\sqrt\{T\})\) regret after \(T\) time steps of agent-environment interaction. We also show that the regret of the proposed algorithm has only a polynomial dependence in the problem dimensions, which gives an exponential improvement over the prior methods. Our improved exploration method is simple, yet efficient, and it combines a sophisticated exploration policy in RL with an isotropic exploration strategy to achieve fast stabilization and improved regret. We empirically demonstrate that the proposed algorithm outperforms other popular

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylale2020reinforcement

Related papers

• Demystifying Reinforcement Learning In Time-varying Systems (2022)0.00
• Learning To Stabilize Online Reinforcement Learning In Unbounded State Spaces (2023)0.00
• Optimal Exploration For Model-based RL In Nonlinear Systems (2023)0.00
• Implications Of Regret On Stability Of Linear Dynamical Systems (2022)6.34
• Sublinear Regret For A Class Of Continuous-time Linear-quadratic Reinforcement Learning Problems (2024)0.00
• Taming "data-hungry" Reinforcement Learning? Stability In Continuous State-action Spaces (2024)2.26
• Continuous Deep Q-learning With Simulator For Stabilization Of Uncertain Discrete-time Systems (2021)0.00
• Logarithmic Regret For Nonlinear Control (2025)0.00