Adapt To Thrive! Adaptive Power-mean Policy Optimization For Improved LLM Reasoning

Abstract

arXiv:2605.04066v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is an essential paradigm that enhances the reasoning capabilities of Large Language Models (LLMs). However, existing methods typically rely on static policy optimization schemes that misalign with the model's evolving reasoning capabilities. To address this issue, we propose Adaptive Power-Mean Policy Optimization (APMPO), which comprises two main innovations: Power-Mean Policy Optimization (PMPO) and Feedback-Adaptive Clipping (FAC). Specifically, PMPO introduces a generalized power-mean objective. This enables the model to adaptively transition from the signal-amplifying behavior of the arithmetic mean to the consistency-enforcing behavior of the geometric mean. FAC adaptively adjusts clipping bounds based on real-time reward statistics to overcome the limitations of static mechanisms. Capitalizing on these innovations, APMPO improves learning dynamics and reasoning performance. E

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhuang2026adapt

Related papers

• Policy Improvement Reinforcement Learning (2026)0.00
• GHPO: Adaptive Guidance For Stable And Efficient LLM Reinforcement Learning (2025)0.00
• Free Energy-driven Reinforcement Learning With Adaptive Advantage Shaping For Unsupervised Reasoning In Llms (2026)0.00
• DVPO: Distributional Value Modeling-based Policy Optimization For LLM Post-training (2026)0.00
• Pairwise Proximal Policy Optimization: Harnessing Relative Feedback For LLM Alignment (2023)0.00
• Think Outside The Policy: In-context Steered Policy Optimization (2025)0.00
• Simple Policy Gradients For Reasoning With Diffusion Language Models (2025)0.00
• Soft Adaptive Policy Optimization (2025)0.00