A General Control-theoretic Approach For Reinforcement Learning: Theory And Algorithms

Abstract

We devise a control-theoretic reinforcement learning approach to support direct learning of the optimal policy. We establish various theoretical properties of our approach, such as convergence and optimality of our analog of the Bellman operator and Q-learning, a new control-policy-variable gradient theorem, and a specific gradient ascent algorithm based on this theorem within the context of a specific control-theoretic framework. We empirically evaluate the performance of our control theoretic approach on several classical reinforcement learning tasks, demonstrating significant improvements in solution quality, sample complexity, and running time of our approach over state-of-the-art methods.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2024a

Related papers

• A General Markov Decision Process Framework For Directly Learning Optimal Control Policies (2019)0.00
• Direct And Indirect Reinforcement Learning (2019)10.74
• Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach (2023)0.00
• Generalized Policy Improvement Algorithms With Theoretically Supported Sample Reuse (2022)5.24
• A Tour Of Reinforcement Learning: The View From Continuous Control (2018)19.86
• Global Convergence Of Policy Gradient Methods In Reinforcement Learning, Games And Control (2023)0.00
• On The Theory Of Policy Gradient Methods: Optimality, Approximation, And Distribution Shift (2019)0.00
• Learning Optimal Deterministic Policies With Stochastic Policy Gradients (2024)0.00