A General Framework For Learning Mean-field Games

Abstract

This paper presents a general mean-field game (GMFG) framework for simultaneous learning and decision-making in stochastic games with a large population. It first establishes the existence of a unique Nash Equilibrium to this GMFG, and demonstrates that naively combining reinforcement learning with the fixed-point approach in classical MFGs yields unstable algorithms. It then proposes value-based and policy-based reinforcement learning algorithms (GMF-V and GMF-P, respectively) with smoothed policies, with analysis of their convergence properties and computational complexities. Experiments on an equilibrium product pricing problem demonstrate that GMF-V-Q and GMF-P-TRPO, two specific instantiations of GMF-V and GMF-P, respectively, with Q-learning and TRPO, are both efficient and robust in the GMFG setting. Moreover, their performance is superior in convergence speed, accuracy, and stability when compared with existing algorithms for multi-agent reinforcement learning in the \(N\)-play

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2020a

Related papers

• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00
• Reinforcement Learning For Mean Field Games With Strategic Complementarities (2020)0.00
• Learning In Mean Field Games: A Survey (2022)0.00
• Generalization In Mean Field Games By Learning Master Policies (2021)7.81
• Policy Optimization For Continuous-time Linear-quadratic Graphon Mean Field Games (2025)0.00
• On The Convergence Of Model Free Learning In Mean Field Games (2019)0.00
• Approximately Solving Mean Field Games Via Entropy-regularized Deep Reinforcement Learning (2021)0.00
• Learning Regularized Graphon Mean-field Games With Unknown Graphons (2023)0.00