On The Variational Interpretation Of Mirror Play In Monotone Games

Abstract

Mirror play (MP) is a well-accepted primal-dual multi-agent learning algorithm where all agents simultaneously implement mirror descent in a distributed fashion. The advantage of MP over vanilla gradient play lies in its usage of mirror maps that better exploit the geometry of decision domains. Despite extensive literature dedicated to the asymptotic convergence of MP to equilibrium, the understanding of the finite-time behavior of MP before reaching equilibrium is still rudimentary. To facilitate the study of MP's non-equilibrium performance, this work establishes an equivalence between MP's finite-time primal-dual path (mirror path) in monotone games and the closed-loop Nash equilibrium path of a finite-horizon differential game, referred to as mirror differential game (MDG). Our construction of MDG rests on the Brezis-Ekeland variational principle, and the stage cost functional for MDG is Fenchel coupling between MP's iterates and associated gradient updates. The variational interpr

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keypan2024on

Related papers

• Multi-agent Online Learning In Time-varying Games (2018)8.82
• Independent Policy Mirror Descent For Markov Potential Games: Scaling To Large Number Of Players (2024)0.00
• Policy Mirror Ascent For Efficient And Independent Learning In Mean Field Games (2022)0.00
• Learning Mirror Maps In Policy Mirror Descent (2024)0.00
• Local And Adaptive Mirror Descents In Extensive-form Games (2023)0.00
• A Unified Approach To Reinforcement Learning, Quantal Response Equilibria, And Two-player Zero-sum Games (2022)0.00
• Policy Mirror Descent With Temporal Difference Learning: Sample Complexity Under Online Markov Data (2025)0.00
• Population-aware Online Mirror Descent For Mean-field Games By Deep Reinforcement Learning (2024)0.00