Learning Equilibria In Mean-field Games: Introducing Mean-field PSRO

Abstract

Recent advances in multiagent learning have seen the introduction ofa family of algorithms that revolve around the population-based trainingmethod PSRO, showing convergence to Nash, correlated and coarse corre-lated equilibria. Notably, when the number of agents increases, learningbest-responses becomes exponentially more difficult, and as such ham-pers PSRO training methods. The paradigm of mean-field games pro-vides an asymptotic solution to this problem when the considered gamesare anonymous-symmetric. Unfortunately, the mean-field approximationintroduces non-linearities which prevent a straightforward adaptation ofPSRO. Building upon optimization and adversarial regret minimization,this paper sidesteps this issue and introduces mean-field PSRO, an adap-tation of PSRO which learns Nash, coarse correlated and correlated equi-libria in mean-field games. The key is to replace the exact distributioncomputation step by newly-defined mean-field no-adversarial-regret learn-ers, or by black

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymuller2021learning

Related papers

• A Generalized Training Approach For Multiagent Learning (2019)0.00
• A General Framework For Learning Mean-field Games (2020)0.00
• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00
• Population-aware Online Mirror Descent For Mean-field Games By Deep Reinforcement Learning (2024)0.00
• Mean Field Multi-agent Reinforcement Learning (2018)2.26
• On The Convergence Of Model Free Learning In Mean Field Games (2019)0.00
• MF-OML: Online Mean-field Reinforcement Learning With Occupation Measures For Large Population Games (2024)3.58
• Neural Population Learning Beyond Symmetric Zero-sum Games (2024)0.00