Learning Regularized Graphon Mean-field Games With Unknown Graphons

Abstract

We design and analyze reinforcement learning algorithms for Graphon Mean-Field Games (GMFGs). In contrast to previous works that require the precise values of the graphons, we aim to learn the Nash Equilibrium (NE) of the regularized GMFGs when the graphons are unknown. Our contributions are threefold. First, we propose the Proximal Policy Optimization for GMFG (GMFG-PPO) algorithm and show that it converges at a rate of \(O(T^\{-1/3\})\) after \(T\) iterations with an estimation oracle, improving on a previous work by Xie et al. (ICML, 2021). Second, using kernel embedding of distributions, we design efficient algorithms to estimate the transition kernels, reward functions, and graphons from sampled agents. Convergence rates are then derived when the positions of the agents are either known or unknown. Results for the combination of the optimization algorithm GMFG-PPO and the estimation algorithm are then provided. These algorithms are the first specifically designed for learning grap

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2023learning

Related papers

• Policy Optimization For Continuous-time Linear-quadratic Graphon Mean Field Games (2025)0.00
• A General Framework For Learning Mean-field Games (2020)0.00
• Approximately Solving Mean Field Games Via Entropy-regularized Deep Reinforcement Learning (2021)0.00
• A Single Online Agent Can Efficiently Learn Mean Field Games (2024)0.00
• Entropy Regularization For Mean Field Games With Learning (2020)0.00
• Generalization In Mean Field Games By Learning Master Policies (2021)7.81
• Reinforcement Learning For Mean Field Games With Strategic Complementarities (2020)0.00
• Learning Equilibria In Mean-field Games: Introducing Mean-field PSRO (2021)0.00