Diffusion Models For Reinforcement Learning: A Survey

Abstract

Diffusion models surpass previous generative models in sample quality and training stability. Recent works have shown the advantages of diffusion models in improving reinforcement learning (RL) solutions. This survey aims to provide an overview of this emerging field and hopes to inspire new avenues of research. First, we examine several challenges encountered by RL algorithms. Then, we present a taxonomy of existing methods based on the roles of diffusion models in RL and explore how the preceding challenges are addressed. We further outline successful applications of diffusion models in various RL-related tasks. Finally, we conclude the survey and offer insights into future research directions. We are actively maintaining a GitHub repository for papers and other related resources in utilizing diffusion models in RL: https://github.com/apexrl/Diff4RLSurvey.

Stats

• citations0
• S2 citations—
• github stars663
• HF likes0
• heat score5.64
• arxiv keyzhu2023diffusion

Code

• apexrl/Diff4RLSurvey—

Related papers

• Policy Representation Via Diffusion Probability Model For Reinforcement Learning (2023)0.00
• Learning From Random Demonstrations: Offline Reinforcement Learning With Importance-sampled Diffusion Models (2024)0.00
• Diffusionnft: Online Diffusion Reinforcement With Forward Process (2025)0.00
• Diffusion Policy Through Conditional Proximal Policy Optimization (2026)0.00
• How Does The Lagrangian Guide Safe Reinforcement Learning Through Diffusion Models? (2026)0.00
• Diffusion Policies As An Expressive Policy Class For Offline Reinforcement Learning (2022)0.00
• Advantage-guided Diffusion For Model-based Reinforcement Learning (2026)0.00
• Understanding Sampler Stochasticity In Training Diffusion Models For RLHF (2025)0.00