DGPO: Distribution Guided Policy Optimization For Fine Grained Credit Assignment

Abstract

arXiv:2605.03327v1 Announce Type: cross Abstract: Reinforcement learning is crucial for aligning large language models to perform complex reasoning tasks. However, current algorithms such as Group Relative Policy Optimization suffer from coarse grained, sequence level credit assignment, which severely struggles to isolate pivotal reasoning steps within long Chain of Thought generations. Furthermore, the standard unbounded Kullback Leibler divergence penalty induces severe gradient instability and mode seeking conservatism, ultimately stifling the discovery of novel reasoning trajectories. To overcome these limitations, we introduce Distribution Guided Policy Optimization, a novel critic free reinforcement learning framework that reinterprets distribution deviation as a guiding signal rather than a rigid penalty.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjin2026dgpo

Related papers

• DVPO: Distributional Value Modeling-based Policy Optimization For LLM Post-training (2026)0.00
• EP-GRPO: Entropy-progress Aligned Group Relative Policy Optimization With Implicit Process Guidance (2026)0.00
• Stepwise Guided Policy Optimization: Coloring Your Incorrect Reasoning In GRPO (2025)0.00
• DGPO: Discovering Multiple Strategies With Diversity-guided Policy Optimization (2022)2.26
• Simple Policy Gradients For Reasoning With Diffusion Language Models (2025)0.00
• Think Outside The Policy: In-context Steered Policy Optimization (2025)0.00
• Cooperative Game-theoretic Credit Assignment For Multi-agent Policy Gradients Via The Core (2025)0.00
• GFT: From Imitation To Reward Fine-tuning With Unbiased Group Advantages And Dynamic Coefficient Rectification (2026)0.00