Real-time Diffusion Policies For Games: Enhancing Consistency Policies With Q-ensembles

Abstract

Diffusion models have shown impressive performance in capturing complex and multi-modal action distributions for game agents, but their slow inference speed prevents practical deployment in real-time game environments. While consistency models offer a promising approach for one-step generation, they often suffer from training instability and performance degradation when applied to policy learning. In this paper, we present CPQE (Consistency Policy with Q-Ensembles), which combines consistency models with Q-ensembles to address these challenges.CPQE leverages uncertainty estimation through Q-ensembles to provide more reliable value function approximations, resulting in better training stability and improved performance compared to classic double Q-network methods. Our extensive experiments across multiple game scenarios demonstrate that CPQE achieves inference speeds of up to 60 Hz -- a significant improvement over state-of-the-art diffusion policies that operate at only 20 Hz -- while

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2025real

Related papers

• Boosting Continuous Control With Consistency Policy (2023)3.58
• Entropy-regularized Diffusion Policy With Q-ensembles For Offline Reinforcement Learning (2024)3.58
• Sampling From Energy-based Policies Using Diffusion (2024)0.00
• Streaming Diffusion Policy: Fast Policy Synthesis With Variable Noise Diffusion Models (2024)0.00
• Contractive Diffusion Policies: Robust Action Diffusion Via Contractive Score-based Sampling With Differential Equations (2026)0.00
• Diffusion Policy Through Conditional Proximal Policy Optimization (2026)0.00
• Learning A Diffusion Model Policy From Rewards Via Q-score Matching (2023)0.00
• Diffusion Policies Creating A Trust Region For Offline Reinforcement Learning (2024)8.04