On Passivity, Reinforcement Learning And Higher-order Learning In Multi-agent Finite Games

Abstract

In this paper, we propose a passivity-based methodology for analysis and design of reinforcement learning in multi-agent finite games. Starting from a known exponentially-discounted reinforcement learning scheme, we show that convergence to a Nash distribution can be shown in the class of games characterized by the monotonicity property of their (negative) payoff. We further exploit passivity to propose a class of higher-order schemes that preserve convergence properties, can improve the speed of convergence and can even converge in cases whereby their first-order counterpart fail to converge. We demonstrate these properties through numerical simulations for several representative games.

Stats

• citations39
• S2 citations—
• github stars0
• HF likes0
• heat score12.02
• arxiv keygao2018on

Related papers

• Convergence Analysis Of Gradient-based Learning With Non-uniform Learning Rates In Non-cooperative Multi-agent Settings (2019)0.00
• Multi-agent Online Learning In Time-varying Games (2018)8.82
• On The Stability Of Learning In Network Games With Many Players (2024)0.00
• Finite-time Last-iterate Convergence For Multi-agent Learning In Games (2020)0.00
• Multi-agent Reinforcement Learning In Cournot Games (2020)0.00
• A Black-box Approach For Non-stationary Multi-agent Reinforcement Learning (2023)0.00
• Game Theory And Multi-agent Reinforcement Learning : From Nash Equilibria To Evolutionary Dynamics (2024)0.00
• From Game-theoretic Multi-agent Log Linear Learning To Reinforcement Learning (2018)0.00