Generalization In Mean Field Games By Learning Master Policies

Abstract

Mean Field Games (MFGs) can potentially scale multi-agent systems to extremely large populations of agents. Yet, most of the literature assumes a single initial distribution for the agents, which limits the practical applications of MFGs. Machine Learning has the potential to solve a wider diversity of MFG problems thanks to generalizations capacities. We study how to leverage these generalization properties to learn policies enabling a typical agent to behave optimally against any population distribution. In reference to the Master equation in MFGs, we coin the term ``Master policies'' to describe them and we prove that a single Master policy provides a Nash equilibrium, whatever the initial distribution. We propose a method to learn such Master policies. Our approach relies on three ingredients: adding the current population distribution as part of the observation, approximating Master policies with neural networks, and training via Reinforcement Learning and Fictitious Play. We illu

Stats

• citations10
• S2 citations—
• github stars0
• HF likes0
• heat score7.81
• arxiv keyperrin2021generalization

Related papers

• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00
• A General Framework For Learning Mean-field Games (2020)0.00
• Networked Communication For Decentralised Agents In Mean-field Games (2023)0.00
• On The Convergence Of Model Free Learning In Mean Field Games (2019)0.00
• Learning In Mean Field Games: A Survey (2022)0.00
• Population-aware Online Mirror Descent For Mean-field Games By Deep Reinforcement Learning (2024)0.00
• Reinforcement Learning For Mean Field Games With Strategic Complementarities (2020)0.00
• Decentralized Mean Field Games (2021)3.58