Higher-order Uncoupled Dynamics Do Not Lead To Nash Equilibrium -- Except When They Do

Abstract

The framework of multi-agent learning explores the dynamics of how individual agent strategies evolve in response to the evolving strategies of other agents. Of particular interest is whether or not agent strategies converge to well known solution concepts such as Nash Equilibrium (NE). Most "fixed order" learning dynamics restrict an agent's underlying state to be its own strategy. In "higher order" learning, agent dynamics can include auxiliary states that can capture phenomena such as path dependencies. We introduce higher-order gradient play dynamics that resemble projected gradient ascent with auxiliary states. The dynamics are "payoff based" in that each agent's dynamics depend on its own evolving payoff. While these payoffs depend on the strategies of other agents in a game setting, agent dynamics do not depend explicitly on the nature of the game or the strategies of other agents. In this sense, dynamics are "uncoupled" since an agent's dynamics do not depend explicitly on the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytoonsi2023higher

Related papers

• Higher-order Uncoupled Learning Dynamics And Nash Equilibrium (2025)0.00
• Learning In Multi-memory Games Triggers Complex Dynamics Diverging From Nash Equilibrium (2023)0.00
• Beyond Strict Competition: Approximate Convergence Of Multi Agent Q-learning Dynamics (2023)0.00
• On The Stability Of Learning In Network Games With Many Players (2024)0.00
• Convergence Analysis Of Gradient-based Learning With Non-uniform Learning Rates In Non-cooperative Multi-agent Settings (2019)0.00
• Game-theoretical Perspectives On Active Equilibria: A Preferred Solution Concept Over Nash Equilibria (2022)0.00
• Multi-agent Online Learning In Time-varying Games (2018)8.82
• Convergence Of Heterogeneous Learning Dynamics In Zero-sum Stochastic Games (2023)2.26