Actor-critic Provably Finds Nash Equilibria Of Linear-quadratic Mean-field Games

Abstract

We study discrete-time mean-field Markov games with infinite numbers of agents where each agent aims to minimize its ergodic cost. We consider the setting where the agents have identical linear state transitions and quadratic cost functions, while the aggregated effect of the agents is captured by the population mean of their states, namely, the mean-field state. For such a game, based on the Nash certainty equivalence principle, we provide sufficient conditions for the existence and uniqueness of its Nash equilibrium. Moreover, to find the Nash equilibrium, we propose a mean-field actor-critic algorithm with linear function approximation, which does not require knowing the model of dynamics. Specifically, at each iteration of our algorithm, we use the single-agent actor-critic algorithm to approximately obtain the optimal policy of the each agent given the current mean-field state, and then update the mean-field state. In particular, we prove that our algorithm converges to the Nash e

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfu2019actor

Related papers

• Reinforcement Learning In Non-stationary Discrete-time Linear-quadratic Mean-field Games (2020)10.07
• Efficiently Computing Nash Equilibria In Adversarial Team Markov Games (2022)0.00
• Mean Field Multi-agent Reinforcement Learning (2018)2.26
• Can We Find Nash Equilibria At A Linear Rate In Markov Games? (2023)0.00
• Computing And Learning Stationary Mean Field Equilibria With Scalar Interactions: Algorithms And Applications (2025)0.00
• Actor-dual-critic Dynamics For Zero-sum And Identical-interest Stochastic Games (2026)0.00
• Linear-quadratic Mean-field Reinforcement Learning: Convergence Of Policy Gradient Methods (2019)0.00
• Networked Communication For Mean-field Games With Function Approximation And Empirical Mean-field Estimation (2024)0.00