Provable Sim-to-real Transfer In Continuous Domain With Partial Observations

Abstract

Sim-to-real transfer trains RL agents in the simulated environments and then deploys them in the real world. Sim-to-real transfer has been widely used in practice because it is often cheaper, safer and much faster to collect samples in simulation than in the real world. Despite the empirical success of the sim-to-real transfer, its theoretical foundation is much less understood. In this paper, we study the sim-to-real transfer in continuous domain with partial observations, where the simulated environments and real-world environments are modeled by linear quadratic Gaussian (LQG) systems. We show that a popular robust adversarial training algorithm is capable of learning a policy from the simulated environment that is competitive to the optimal policy in the real-world environment. To achieve our results, we design a new algorithm for infinite-horizon average-cost LQGs and establish a regret bound that depends on the intrinsic complexity of the model class. Our algorithm crucially reli

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhu2022provable

Related papers

• Learning Robust And Adaptive Real-world Continuous Control Using Simulation And Transfer Learning (2018)0.00
• Understanding Domain Randomization For Sim-to-real Transfer (2021)0.00
• Overcoming The Sim-to-real Gap: Leveraging Simulation To Learn To Explore For Real-world RL (2024)5.84
• Post-convergence Sim-to-real Policy Transfer: A Principled Alternative To Cherry-picking (2025)0.00
• Transfer Learning Across Simulated Robots With Different Sensors (2019)0.00
• Learning The Linear Quadratic Regulator From Nonlinear Observations (2020)0.00
• Continuous Deep Q-learning With Simulator For Stabilization Of Uncertain Discrete-time Systems (2021)0.00
• Sublinear Regret For A Class Of Continuous-time Linear-quadratic Reinforcement Learning Problems (2024)0.00