Equilibrium Selection For Multi-agent Reinforcement Learning: A Unified Framework

Abstract

While multi-agent reinforcement learning (MARL) has produced numerous algorithms that converge to Nash or related equilibria, such equilibria are often non-unique and can exhibit widely varying efficiency. This raises a fundamental question: how can one design learning dynamics that not only converge to equilibrium but also select equilibria with desirable performance, such as high social welfare? In contrast to the MARL literature, equilibrium selection has been extensively studied in normal-form games, where decentralized dynamics are known to converge to potential-maximizing or Pareto-optimal Nash equilibria (NEs). Motivated by these results, we study equilibrium selection in finite-horizon stochastic games. We propose a unified actor-critic framework in which a critic learns state-action value functions, and an actor applies a classical equilibrium-selection rule state-wise, treating learned values as stage-game payoffs. We show that, under standard stochastic stability assumptio

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2024equilibrium

Related papers

• Bi-level Actor-critic For Multi-agent Coordination (2019)0.00
• Game-theoretical Perspectives On Active Equilibria: A Preferred Solution Concept Over Nash Equilibria (2022)0.00
• Game Theory And Multi-agent Reinforcement Learning : From Nash Equilibria To Evolutionary Dynamics (2024)0.00
• Inducing Stackelberg Equilibrium Through Spatio-temporal Sequential Decision-making In Multi-agent Reinforcement Learning (2023)7.50
• Maximum Entropy Heterogeneous-agent Reinforcement Learning (2023)0.00
• Faster Last-iterate Convergence Of Policy Optimization In Zero-sum Markov Games (2022)0.00
• Specification-guided Learning Of Nash Equilibria With High Social Welfare (2022)0.00
• Safe Multi-agent Reinforcement Learning With Convergence To Generalized Nash Equilibrium (2024)0.00