Approximately Solving Mean Field Games Via Entropy-regularized Deep Reinforcement Learning

Abstract

The recent mean field game (MFG) formalism facilitates otherwise intractable computation of approximate Nash equilibria in many-agent settings. In this paper, we consider discrete-time finite MFGs subject to finite-horizon objectives. We show that all discrete-time finite MFGs with non-constant fixed point operators fail to be contractive as typically assumed in existing MFG literature, barring convergence via fixed point iteration. Instead, we incorporate entropy-regularization and Boltzmann policies into the fixed point iteration. As a result, we obtain provable convergence to approximate fixed points where existing methods fail, and reach the original goal of approximate Nash equilibria. All proposed methods are evaluated with respect to their exploitability, on both instructive examples with tractable exact solutions and high-dimensional problems where exact methods become intractable. In high-dimensional scenarios, we apply established deep reinforcement learning methods and empir

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycui2021approximately

Related papers

• Entropy Regularization For Mean Field Games With Learning (2020)0.00
• Reinforcement Learning For Mean Field Games With Strategic Complementarities (2020)0.00
• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00
• A General Framework For Learning Mean-field Games (2020)0.00
• Maximum Causal Entropy IRL In Mean-field Games And GNEP Framework For Forward RL (2024)0.00
• Networked Communication For Decentralised Agents In Mean-field Games (2023)0.00
• Oracle-free Reinforcement Learning In Mean-field Games Along A Single Sample Path (2022)0.00
• Efficient And Scalable Deep Reinforcement Learning For Mean Field Control Games (2024)0.00