Multi-agent Training Beyond Zero-sum With Correlated Equilibrium Meta-solvers

Abstract

Two-player, constant-sum games are well studied in the literature, but there has been limited progress outside of this setting. We propose Joint Policy-Space Response Oracles (JPSRO), an algorithm for training agents in n-player, general-sum extensive form games, which provably converges to an equilibrium. We further suggest correlated equilibria (CE) as promising meta-solvers, and propose a novel solution concept Maximum Gini Correlated Equilibrium (MGCE), a principled and computationally efficient family of solutions for solving the correlated equilibrium selection problem. We conduct several experiments using CE meta-solvers for JPSRO and demonstrate convergence on n-player, general-sum games.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymarris2021multi

Related papers

• A Generalized Training Approach For Multiagent Learning (2019)0.00
• Neural Population Learning Beyond Symmetric Zero-sum Games (2024)0.00
• Fictitious Cross-play: Learning Global Nash Equilibrium In Mixed Cooperative-competitive Games (2023)3.58
• Faster Last-iterate Convergence Of Policy Optimization In Zero-sum Markov Games (2022)0.00
• Calibration Of Shared Equilibria In General Sum Partially Observable Markov Games (2020)0.00
• Policy Optimization For Markov Games: Unified Framework And Faster Convergence (2022)0.00
• Generative Evolutionary Meta-solver (GEMS): Scalable Surrogate-free Multi-agent Reinforcement Learning (2025)0.00
• Learning Equilibria In Mean-field Games: Introducing Mean-field PSRO (2021)0.00