Neural Population Learning Beyond Symmetric Zero-sum Games

Abstract

We study computationally efficient methods for finding equilibria in n-player general-sum games, specifically ones that afford complex visuomotor skills. We show how existing methods would struggle in this setting, either computationally or in theory. We then introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated Equilibrium (CCE) of the game. We show empirical convergence in a suite of OpenSpiel games, validated rigorously by exact game solvers. We then deploy NeuPL-JPSRO to complex domains, where our approach enables adaptive coordination in a MuJoCo control domain and skill transfer in capture-the-flag. Our work shows that equilibrium convergent population learning can be implemented at scale and in generality, paving the way towards solving real-world games between heterogeneous players with mixed motives.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliu2024neural

Related papers

• Neupl: Neural Population Learning (2022)0.00
• Multi-agent Training Beyond Zero-sum With Correlated Equilibrium Meta-solvers (2021)0.00
• Learning Equilibria In Mean-field Games: Introducing Mean-field PSRO (2021)0.00
• Simplex Neural Population Learning: Any-mixture Bayes-optimality In Symmetric Zero-sum Games (2022)0.00
• A Generalized Training Approach For Multiagent Learning (2019)0.00
• Fictitious Cross-play: Learning Global Nash Equilibrium In Mixed Cooperative-competitive Games (2023)3.58
• Near Optimal Convergence To Coarse Correlated Equilibrium In General-sum Markov Games (2025)0.00
• Pipeline PSRO: A Scalable Approach For Finding Approximate Nash Equilibria In Large Games (2020)0.00