Bridging Imagination And Reality For Model-based Deep Reinforcement Learning

Abstract

Sample efficiency has been one of the major challenges for deep reinforcement learning. Recently, model-based reinforcement learning has been proposed to address this challenge by performing planning on imaginary trajectories with a learned world model. However, world model learning may suffer from overfitting to training trajectories, and thus model-based value estimation and policy search will be pone to be sucked in an inferior local policy. In this paper, we propose a novel model-based reinforcement learning algorithm, called BrIdging Reality and Dream (BIRD). It maximizes the mutual information between imaginary and real trajectories so that the policy improvement learned from imaginary trajectories can be easily generalized to real trajectories. We demonstrate that our approach improves sample efficiency of model-based planning, and achieves state-of-the-art performance on challenging visual control benchmarks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhu2020bridging

Related papers

• Harmonydream: Task Harmonization Inside World Models (2023)3.46
• Acting Upon Imagination: When To Trust Imagined Trajectories In Model Based Reinforcement Learning (2021)0.00
• Dream To Control: Learning Behaviors By Latent Imagination (2019)0.00
• Imagine-2-drive: Leveraging High-fidelity World Models Via Multi-modal Diffusion Policies (2024)0.00
• Towards Biologically Plausible Dreaming And Planning In Recurrent Spiking Networks (2022)0.00
• Learning To Reweight Imaginary Transitions For Model-based Reinforcement Learning (2021)0.00
• Simplifying Model-based RL: Learning Representations, Latent-space Models, And Policies With One Objective (2022)0.00
• Synthesizing World Models For Bilevel Planning (2025)0.00