A Tour Of Reinforcement Learning: The View From Continuous Control

Abstract

This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. It surveys the general formulation, terminology, and typical experimental implementations of reinforcement learning and reviews competing solution paradigms. In order to compare the relative merits of various techniques, this survey presents a case study of the Linear Quadratic Regulator (LQR) with unknown dynamics, perhaps the simplest and best-studied problem in optimal control. The manuscript describes how merging techniques from learning theory and control can provide non-asymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. Thi

Stats

• citations443
• S2 citations—
• github stars0
• HF likes0
• heat score19.86
• arxiv keyrecht2018a

Related papers

• On The Optimization Landscape Of Dynamic Output Feedback: A Case Study For Linear Quadratic Regulator (2022)4.52
• Sublinear Regret For A Class Of Continuous-time Linear-quadratic Reinforcement Learning Problems (2024)0.00
• Learning The Linear Quadratic Regulator From Nonlinear Observations (2020)0.00
• Robust Reinforcement Learning: A Case Study In Linear Quadratic Regulation (2020)11.19
• Least-squares Temporal Difference Learning For The Linear Quadratic Regulator (2017)0.00
• RLOC: Neurobiologically Inspired Hierarchical Reinforcement Learning Algorithm For Continuous Control Of Nonlinear Dynamical Systems (2019)0.00
• An Efficient Off-policy Reinforcement Learning Algorithm For The Continuous-time LQR Problem (2023)6.34
• Some Remarks On Gradient Dominance And LQR Policy Optimization (2025)0.00