A Single Online Agent Can Efficiently Learn Mean Field Games

Abstract

Mean field games (MFGs) are a promising framework for modeling the behavior of large-population systems. However, solving MFGs can be challenging due to the coupling of forward population evolution and backward agent dynamics. Typically, obtaining mean field Nash equilibria (MFNE) involves an iterative approach where the forward and backward processes are solved alternately, known as fixed-point iteration (FPI). This method requires fully observed population propagation and agent dynamics over the entire spatial domain, which could be impractical in some real-world scenarios. To overcome this limitation, this paper introduces a novel online single-agent model-free learning scheme, which enables a single agent to learn MFNE using online samples, without prior knowledge of the state-action space, reward function, or transition dynamics. Specifically, the agent updates its policy through the value function (Q), while simultaneously evaluating the mean field state (M), using the same batch

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2024a

Related papers

• On The Convergence Of Model Free Learning In Mean Field Games (2019)0.00
• Networked Communication For Decentralised Agents In Mean-field Games (2023)0.00
• Generalization In Mean Field Games By Learning Master Policies (2021)7.81
• Oracle-free Reinforcement Learning In Mean-field Games Along A Single Sample Path (2022)0.00
• A General Framework For Learning Mean-field Games (2020)0.00
• Population-aware Online Mirror Descent For Mean-field Games By Deep Reinforcement Learning (2024)0.00
• Reinforcement Learning For Mean Field Games With Strategic Complementarities (2020)0.00
• Decentralized Mean Field Games (2021)3.58