Phasic Diversity Optimization For Population-based Reinforcement Learning

Abstract

Reviewing the previous work of diversity Rein-forcement Learning,diversity is often obtained via an augmented loss function,which requires a balance between reward and diversity.Generally,diversity optimization algorithms use Multi-armed Bandits algorithms to select the coefficient in the pre-defined space. However, the dynamic distribution of reward signals for MABs or the conflict between quality and diversity limits the performance of these methods. We introduce the Phasic Diversity Optimization (PDO) algorithm, a Population-Based Training framework that separates reward and diversity training into distinct phases instead of optimizing a multi-objective function. In the auxiliary phase, agents with poor performance diversified via determinants will not replace the better agents in the archive. The decoupling of reward and diversity allows us to use an aggressive diversity optimization in the auxiliary phase without performance degradation. Furthermore, we construct a dogfight scenar

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjiang2024phasic

Related papers

• Effective Diversity In Population Based Reinforcement Learning (2020)0.00
• DGPO: Discovering Multiple Strategies With Diversity-guided Policy Optimization (2022)2.26
• Diverse Policies Converge In Reward-free Markov Decision Processe (2023)0.00
• Diversity Policy Gradient For Sample Efficient Quality-diversity Optimization (2020)11.58
• Structured Diversity Control: A Dual-level Framework For Group-aware Multi-agent Coordination (2025)0.00
• Unifying Behavioral And Response Diversity For Open-ended Learning In Zero-sum Games (2021)0.00
• Selection-expansion: A Unifying Framework For Motion-planning And Diversity Search Algorithms (2021)0.00
• Social Diversity And Social Preferences In Mixed-motive Reinforcement Learning (2020)0.00