Quantifying The Effects Of Environment And Population Diversity In Multi-agent Reinforcement Learning

Abstract

Generalization is a major challenge for multi-agent reinforcement learning. How well does an agent perform when placed in novel environments and in interactions with new co-players? In this paper, we investigate and quantify the relationship between generalization and diversity in the multi-agent domain. Across the range of multi-agent environments considered here, procedurally generating training levels significantly improves agent performance on held-out levels. However, agent performance on the specific levels used in training sometimes declines as a result. To better understand the effects of co-player variation, our experiments introduce a new environment-agnostic measure of behavioral diversity. Results demonstrate that population size and intrinsic motivation are both effective methods of generating greater population diversity. In turn, training with a diverse set of co-players strengthens agent performance in some (but not all) cases.

Stats

• citations15
• S2 citations—
• github stars0
• HF likes0
• heat score9.03
• arxiv keymckee2021quantifying

Related papers

• The Impact Of Behavioral Diversity In Multi-agent Reinforcement Learning (2024)0.00
• Effective Diversity In Population Based Reinforcement Learning (2020)0.00
• System Neural Diversity: Measuring Behavioral Heterogeneity In Multi-agent Learning (2023)0.00
• Deep Multiagent Reinforcement Learning: Challenges And Directions (2021)0.00
• Quantifying Agent Interaction In Multi-agent Reinforcement Learning For Cost-efficient Generalization (2023)0.00
• Social Diversity And Social Preferences In Mixed-motive Reinforcement Learning (2020)0.00
• Dynamic Noises Of Multi-agent Environments Can Improve Generalization: Agent-based Models Meets Reinforcement Learning (2022)0.00
• On Generalization Across Environments In Multi-objective Reinforcement Learning (2025)0.00