Trade-off On Sim2real Learning: Real-world Learning Faster Than Simulations

Abstract

Deep Reinforcement Learning (DRL) experiments are commonly performed in simulated environments due to the tremendous training sample demands from deep neural networks. In contrast, model-based Bayesian Learning allows a robot to learn good policies within a few trials in the real world. Although it takes fewer iterations, Bayesian methods pay a relatively higher computational cost per trial, and the advantage of such methods is strongly tied to dimensionality and noise. In here, we compare a Deep Bayesian Learning algorithm with a model-free DRL algorithm while analyzing our results collected from both simulations and real-world experiments. While considering Sim and Real learning, our experiments show that the sample-efficient Deep Bayesian RL performance is better than DRL even when computation time (as opposed to number of iterations) is taken in consideration. Additionally, the difference in computation time between Deep Bayesian RL performed in simulation and in experiments point

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyhuang2020trade

Related papers

• Overcoming The Sim-to-real Gap: Leveraging Simulation To Learn To Explore For Real-world RL (2024)5.84
• Post-convergence Sim-to-real Policy Transfer: A Principled Alternative To Cherry-picking (2025)0.00
• Efficient Exploration In Deep Reinforcement Learning: A Novel Bayesian Actor-critic Algorithm (2024)0.00
• On The Mistaken Assumption Of Interchangeable Deep Reinforcement Learning Implementations (2025)0.00
• Influence-augmented Local Simulators: A Scalable Solution For Fast Deep RL In Large Networked Systems (2022)0.00
• Towards Model-based Reinforcement Learning For Industry-near Environments (2019)5.84
• Learning Curricula In Open-ended Worlds (2023)0.00
• Human-inspired Framework To Accelerate Reinforcement Learning (2023)0.00