Cost-driven Representation Learning For Linear Quadratic Gaussian Control: Part I

Abstract

We study the task of learning state representations from potentially high-dimensional observations, with the goal of controlling an unknown partially observable system. We pursue a cost-driven approach, where a dynamic model in some latent state space is learned by predicting the costs without predicting the observations or actions. In particular, we focus on an intuitive cost-driven state representation learning method for solving Linear Quadratic Gaussian (LQG) control, one of the most fundamental partially observable control problems. As our main results, we establish finite-sample guarantees of finding a near-optimal state representation function and a near-optimal controller using the directly learned latent model, for finite-horizon time-varying LQG control problems. To the best of our knowledge, despite various empirical successes, finite-sample guarantees of such a cost-driven approach remain elusive. Our result underscores the value of predicting multi-step costs, an idea that

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytian2022cost

Related papers

• Learning The Linear Quadratic Regulator From Nonlinear Observations (2020)0.00
• Online Policy Gradient For Model Free Learning Of Linear Quadratic Regulators With \(\sqrt{t}\) Regret (2021)0.00
• Extracting Latent State Representations With Linear Dynamics From Rich Observations (2020)0.00
• Sublinear Regret For A Class Of Continuous-time Linear-quadratic Reinforcement Learning Problems (2024)0.00
• Fast Policy Learning For Linear Quadratic Control With Entropy Regularization (2023)0.00
• Revisiting LQR Control From The Perspective Of Receding-horizon Policy Gradient (2023)8.60
• Distributed Q-learning With State Tracking For Multi-agent Networked Control (2020)0.00
• Deep Q-learning: A Robust Control Approach (2022)9.23