Imagine-2-drive: Leveraging High-fidelity World Models Via Multi-modal Diffusion Policies

Abstract

World Model-based Reinforcement Learning (WMRL) enables sample efficient policy learning by reducing the need for online interactions which can potentially be costly and unsafe, especially for autonomous driving. However, existing world models often suffer from low prediction fidelity and compounding one-step errors, leading to policy degradation over long horizons. Additionally, traditional RL policies, often deterministic or single Gaussian-based, fail to capture the multi-modal nature of decision-making in complex driving scenarios. To address these challenges, we propose Imagine-2-Drive, a novel WMRL framework that integrates a high-fidelity world model with a multi-modal diffusion-based policy actor. It consists of two key components: DiffDreamer, a diffusion-based world model that generates future observations simultaneously, mitigating error accumulation, and DPA (Diffusion Policy Actor), a diffusion-based policy that models diverse and multi-modal trajectory distributions. By t

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygarg2024imagine

Related papers

• Diwa: Diffusion Policy Adaptation With World Models (2025)0.00
• World Models Via Policy-guided Trajectory Diffusion (2023)0.00
• Diffusion World Model: Future Modeling Beyond Step-by-step Rollout For Offline Reinforcement Learning (2024)0.00
• PWM: Policy Learning With Multi-task World Models (2024)0.00
• Enhancing Policy Learning With World-action Model (2026)0.00
• World Models As Reference Trajectories For Rapid Motor Adaptation (2025)0.00
• Steering Your Diffusion Policy With Latent Space Reinforcement Learning (2025)0.00
• Policy Representation Via Diffusion Probability Model For Reinforcement Learning (2023)0.00