Mean Field Multi-agent Reinforcement Learning

Abstract

Existing multi-agent reinforcement learning methods are limited typically to a small number of agents. When the agent number increases largely, the learning becomes intractable due to the curse of the dimensionality and the exponential growth of agent interactions. In this paper, we present *Mean Field Reinforcement Learning* where the interactions within the population of agents are approximated by those between a single agent and the average effect from the overall population or neighboring agents; the interplay between the two entities is mutually reinforced: the learning of the individual agent's optimal policy depends on the dynamics of the population, while the dynamics of the population change according to the collective patterns of the individual policies. We develop practical mean field Q-learning and mean field Actor-Critic algorithms and analyze the convergence of the solution to Nash equilibrium. Experiments on Gaussian squeeze, Ising model, and battle games justify the lea

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyyang2018mean

Related papers

• Partially Observable Mean Field Reinforcement Learning (2020)0.00
• Decentralized Mean Field Games (2021)3.58
• Multi Type Mean Field Reinforcement Learning (2020)0.00
• Efficient Model-based Multi-agent Mean-field Reinforcement Learning (2021)0.00
• Causal Mean Field Multi-agent Reinforcement Learning (2025)2.26
• On The Convergence Of Model Free Learning In Mean Field Games (2019)0.00
• Partially Observable Mean Field Multi-agent Reinforcement Learning Based On Graph-attention (2023)9.76
• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00