Learning In Multi-memory Games Triggers Complex Dynamics Diverging From Nash Equilibrium

Abstract

Repeated games consider a situation where multiple agents are motivated by their independent rewards throughout learning. In general, the dynamics of their learning become complex. Especially when their rewards compete with each other like zero-sum games, the dynamics often do not converge to their optimum, i.e., the Nash equilibrium. To tackle such complexity, many studies have understood various learning algorithms as dynamical systems and discovered qualitative insights among the algorithms. However, such studies have yet to handle multi-memory games (where agents can memorize actions they played in the past and choose their actions based on their memories), even though memorization plays a pivotal role in artificial intelligence and interpersonal relationship. This study extends two major learning algorithms in games, i.e., replicator dynamics and gradient ascent, into multi-memory games. Then, we prove their dynamics are identical. Furthermore, theoretically and experimentally, we

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfujimoto2023learning

Related papers

• Memory Asymmetry Creates Heteroclinic Orbits To Nash Equilibrium In Learning In Zero-sum Games (2023)0.00
• On The Stability Of Learning In Network Games With Many Players (2024)0.00
• Synchronization In Learning In Periodic Zero-sum Games Triggers Divergence From Nash Equilibrium (2024)0.00
• Higher-order Uncoupled Dynamics Do Not Lead To Nash Equilibrium -- Except When They Do (2023)0.00
• Asymptotic Convergence And Performance Of Multi-agent Q-learning Dynamics (2023)0.00
• Chaos Persists In Large-scale Multi-agent Learning Despite Adaptive Learning Rates (2023)0.00
• Convergence Of Heterogeneous Learning Dynamics In Zero-sum Stochastic Games (2023)2.26
• Game Theory And Multi-agent Reinforcement Learning : From Nash Equilibria To Evolutionary Dynamics (2024)0.00