Learning Large-scale Competitive Team Behaviors With Mean-field Interactions And Online Opponent Modeling

Abstract

While multi-agent reinforcement learning (MARL) has been proven effective across both collaborative and competitive tasks, existing algorithms often struggle to scale to large populations of agents. Recent advancements in mean-field (MF) theory provide scalable solutions by approximating population interactions as a continuum, yet most existing frameworks focus exclusively on either fully cooperative or purely competitive settings. To bridge this gap, we introduce MF-MAPPO, a mean-field extension of PPO designed for zero-sum team games that integrate intra-team cooperation with inter-team competition. MF-MAPPO employs a shared actor and a minimally informed critic per team and is trained directly on finite-population simulators, thereby enabling deployment to realistic scenarios with thousands of agents. We further show that MF-MAPPO naturally extends to partially observable settings through a simple gradient-regularized training scheme. Our evaluation utilizes large-scale benchmark sc

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjeloka2025learning

Related papers

• MF-OML: Online Mean-field Reinforcement Learning With Occupation Measures For Large Population Games (2024)3.58
• Cooperation Dynamics In Multi-agent Systems: Exploring Game-theoretic Scenarios With Mean-field Equilibria (2023)0.00
• Partially Observable Mean Field Multi-agent Reinforcement Learning Based On Graph-attention (2023)9.76
• Heterogeneous Multi-agent Reinforcement Learning For Zero-shot Scalable Collaboration (2024)6.34
• Mean Field Multi-agent Reinforcement Learning (2018)2.26
• Permutation Invariant Policy Optimization For Mean-field Multi-agent Reinforcement Learning: A Principled Approach (2021)0.00
• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00
• Language-driven Coordination And Learning In Multi-agent Simulation Environments (2025)0.00