MF-OML: Online Mean-field Reinforcement Learning With Occupation Measures For Large Population Games

Abstract

Reinforcement learning for multi-agent games has attracted lots of attention recently. However, given the challenge of solving Nash equilibria for large population games, existing works with guaranteed polynomial complexities either focus on variants of zero-sum and potential games, or aim at solving (coarse) correlated equilibria, or require access to simulators, or rely on certain assumptions that are hard to verify. This work proposes MF-OML (Mean-Field Occupation-Measure Learning), an online mean-field reinforcement learning algorithm for computing approximate Nash equilibria of large population sequential symmetric games. MF-OML is the first fully polynomial multi-agent reinforcement learning algorithm for provably solving Nash equilibria (up to mean-field approximation gaps that vanish as the number of players \(N\) goes to infinity) beyond variants of zero-sum and potential games. When evaluated by the cumulative deviation from Nash equilibria, the algorithm is shown to achieve

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyhu2024mf

Related papers

• Learning Large-scale Competitive Team Behaviors With Mean-field Interactions And Online Opponent Modeling (2025)0.00
• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00
• A General Framework For Learning Mean-field Games (2020)0.00
• Learning In Mean Field Games: A Survey (2022)0.00
• Population-aware Online Mirror Descent For Mean-field Games By Deep Reinforcement Learning (2024)0.00
• Networked Communication For Decentralised Agents In Mean-field Games (2023)0.00
• Scalable Offline Reinforcement Learning For Mean Field Games (2024)0.00
• Learning Equilibria In Mean-field Games: Introducing Mean-field PSRO (2021)0.00