Entropy Regularization For Mean Field Games With Learning

Abstract

Entropy regularization has been extensively adopted to improve the efficiency, the stability, and the convergence of algorithms in reinforcement learning. This paper analyzes both quantitatively and qualitatively the impact of entropy regularization for Mean Field Game (MFG) with learning in a finite time horizon. Our study provides a theoretical justification that entropy regularization yields time-dependent policies and, furthermore, helps stabilizing and accelerating convergence to the game equilibrium. In addition, this study leads to a policy-gradient algorithm for exploration in MFG. Under this algorithm, agents are able to learn the optimal exploration scheduling, with stable and fast convergence to the game equilibrium.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2020entropy

Related papers

• Approximately Solving Mean Field Games Via Entropy-regularized Deep Reinforcement Learning (2021)0.00
• Marginalized State Distribution Entropy Regularization In Policy Optimization (2019)0.00
• Q-learning In Regularized Mean-field Games (2020)0.00
• Policy Optimization Finds Nash Equilibrium In Regularized General-sum LQ Games (2024)0.00
• Linear Convergence Of Independent Natural Policy Gradient In Games With Entropy Regularization (2024)3.58
• Entropy Regularized Reinforcement Learning Using Large Deviation Theory (2021)6.34
• Understanding The Impact Of Entropy On Policy Optimization (2018)0.00
• Fast Policy Extragradient Methods For Competitive Games With Entropy Regularization (2021)0.00