Policy Gradient For LQR With Domain Randomization

Abstract

Domain randomization (DR) enables sim-to-real transfer by training controllers on a distribution of simulated environments, with the goal of achieving robust performance in the real world. Although DR is widely used in practice and is often solved using simple policy gradient (PG) methods, understanding of its theoretical guarantees remains limited. Toward addressing this gap, we provide the first convergence analysis of PG methods for domain-randomized linear quadratic regulation (LQR). We show that PG converges globally to the minimizer of a finite-sample approximation of the DR objective under suitable bounds on the heterogeneity of the sampled systems. We also quantify the sample-complexity associated with achieving a small performance gap between the sample-average and population-level objectives. Additionally, we propose and analyze a discount-factor annealing algorithm that obviates the need for an initial jointly stabilizing controller, which may be challenging to find. Empiric

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyfujinami2025policy

Related papers

• Revisiting LQR Control From The Perspective Of Receding-horizon Policy Gradient (2023)8.60
• Full Error Analysis Of Policy Gradient Learning Algorithms For Exploratory Linear Quadratic Mean-field Control Problem In Continuous Time With Common Noise (2024)0.00
• Some Remarks On Gradient Dominance And LQR Policy Optimization (2025)0.00
• Learning Robust Control For LQR Systems With Multiplicative Noise Via Policy Gradient (2019)0.00
• Convergence Of Policy Gradient Methods For Finite-horizon Exploratory Linear-quadratic Control Problems (2022)9.23
• Meta-learning Linear Quadratic Regulators: A Policy Gradient MAML Approach For Model-free LQR (2024)0.00
• Oracle Complexity Reduction For Model-free LQR: A Stochastic Variance-reduced Policy Gradient Approach (2023)2.26
• Implicit Bias Of Policy Gradient In Linear Quadratic Control: Extrapolation To Unseen Initial States (2024)0.00