Deep Reinforcement Learning For Infinite Horizon Mean Field Problems In Continuous Spaces

Abstract

We present the development and analysis of a reinforcement learning (RL) algorithm designed to solve continuous-space mean field game (MFG) and mean field control (MFC) problems in a unified manner. The proposed approach pairs the actor-critic (AC) paradigm with a representation of the mean field distribution via a parameterized score function, which can be efficiently updated in an online fashion, and uses Langevin dynamics to obtain samples from the resulting distribution. The AC agent and the score function are updated iteratively to converge, either to the MFG equilibrium or the MFC optimum for a given mean field problem, depending on the choice of learning rates. A straightforward modification of the algorithm allows us to solve mixed mean field control games (MFCGs). The performance of our algorithm is evaluated using linear-quadratic benchmarks in the asymptotic infinite horizon framework.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyangiuli2023deep

Related papers

• Unified Reinforcement Q-learning For Mean Field Game And Control Problems (2020)0.00
• Efficient And Scalable Deep Reinforcement Learning For Mean Field Control Games (2024)0.00
• Convergence Of Actor-critic Learning For Mean Field Games And Mean Field Control In Continuous Spaces (2025)0.00
• Reinforcement Learning In Non-stationary Discrete-time Linear-quadratic Mean-field Games (2020)10.07
• Model-free Mean-field Reinforcement Learning: Mean-field MDP And Mean-field Q-learning (2019)0.00
• Analysis Of Multiscale Reinforcement Q-learning Algorithms For Mean Field Control Games (2024)0.00
• Learning Mean-field Games Through Mean-field Actor-critic Flow (2025)0.00
• Global Convergence Of Policy Gradient For Linear-quadratic Mean-field Control/game In Continuous Time (2020)0.00