Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning

Abstract

Diffusion models generate samples through an iterative denoising process, guided by a neural network. While training the denoiser on real-world data is computationally demanding, the sampling procedure itself is more flexible. This adaptability serves as a key lever in practice, enabling improvements in both the quality of generated samples and the efficiency of the sampling process. In this work, we introduce an inverse reinforcement learning framework for learning sampling strategies without retraining the denoiser. We formulate the diffusion sampling procedure as a discrete-time finite-horizon Markov Decision Process, where actions correspond to optional modifications of the sampling dynamics. To optimize action scheduling, we avoid defining an explicit reward function. Instead, we directly match the target behavior expected from the sampler using policy gradient techniques. We provide experimental evidence that this approach can improve the quality of samples generated by pretraine

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybourdrez2026learning

Related papers

• Diffusionnft: Online Diffusion Reinforcement With Forward Process (2025)0.00
• Using Human Feedback To Fine-tune Diffusion Models Without Any Reward Model (2023)17.39
• Understanding Sampler Stochasticity In Training Diffusion Models For RLHF (2025)0.00
• Learning From Random Demonstrations: Offline Reinforcement Learning With Importance-sampled Diffusion Models (2024)0.00
• Diffusion Policy Through Conditional Proximal Policy Optimization (2026)0.00
• Fine-tuning Diffusion Policies With Backpropagation Through Diffusion Timesteps (2025)0.00
• Avoiding Mode Collapse In Diffusion Models Fine-tuned With Reinforcement Learning (2024)0.00
• Diwa: Diffusion Policy Adaptation With World Models (2025)0.00