Effective Diversity In Population Based Reinforcement Learning

Abstract

Exploration is a key problem in reinforcement learning, since agents can only learn from data they acquire in the environment. With that in mind, maintaining a population of agents is an attractive method, as it allows data be collected with a diverse set of behaviors. This behavioral diversity is often boosted via multi-objective loss functions. However, those approaches typically leverage mean field updates based on pairwise distances, which makes them susceptible to cycling behaviors and increased redundancy. In addition, explicitly boosting diversity often has a detrimental impact on optimizing already fruitful behaviors for rewards. As such, the reward-diversity trade off typically relies on heuristics. Finally, such methods require behavioral representations, often handcrafted and domain specific. In this paper, we introduce an approach to optimize all members of a population simultaneously. Rather than using pairwise distance, we measure the volume of the entire population in a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyparkerholder2020effective

Related papers

• Phasic Diversity Optimization For Population-based Reinforcement Learning (2024)0.00
• Quantifying The Effects Of Environment And Population Diversity In Multi-agent Reinforcement Learning (2021)9.03
• Social Diversity And Social Preferences In Mixed-motive Reinforcement Learning (2020)0.00
• The Impact Of Behavioral Diversity In Multi-agent Reinforcement Learning (2024)0.00
• DGPO: Discovering Multiple Strategies With Diversity-guided Policy Optimization (2022)2.26
• Diverse Policies Converge In Reward-free Markov Decision Processe (2023)0.00
• The Curse Of Diversity In Ensemble-based Exploration (2024)0.00
• Selection-expansion: A Unifying Framework For Motion-planning And Diversity Search Algorithms (2021)0.00