On Imitation In Mean-field Games

Abstract

We explore the problem of imitation learning (IL) in the context of mean-field games (MFGs), where the goal is to imitate the behavior of a population of agents following a Nash equilibrium policy according to some unknown payoff function. IL in MFGs presents new challenges compared to single-agent IL, particularly when both the reward function and the transition kernel depend on the population distribution. In this paper, departing from the existing literature on IL for MFGs, we introduce a new solution concept called the Nash imitation gap. Then we show that when only the reward depends on the population distribution, IL in MFGs can be reduced to single-agent IL with similar guarantees. However, when the dynamics is population-dependent, we provide a novel upper-bound that suggests IL is harder in this setting. To address this issue, we propose a new adversarial formulation where the reinforcement learning problem is replaced by a mean-field control (MFC) problem, suggesting progress

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyramponi2023on

Related papers

• Mean Field Correlated Imitation Learning (2024)0.00
• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00
• Adversarial Inverse Reinforcement Learning For Mean Field Games (2021)0.00
• A General Framework For Learning Mean-field Games (2020)0.00
• Reinforcement Learning For Mean Field Games With Strategic Complementarities (2020)0.00
• Generalization In Mean Field Games By Learning Master Policies (2021)7.81
• Learning In Mean Field Games: A Survey (2022)0.00
• Population-aware Online Mirror Descent For Mean-field Games By Deep Reinforcement Learning (2024)0.00