The Evolutionary Dynamics Of Independent Learning Agents In Population Games

Abstract

Understanding the evolutionary dynamics of reinforcement learning under multi-agent settings has long remained an open problem. While previous works primarily focus on 2-player games, we consider population games, which model the strategic interactions of a large population comprising small and anonymous agents. This paper presents a formal relation between stochastic processes and the dynamics of independent learning agents who reason based on the reward signals. Using a master equation approach, we provide a novel unified framework for characterising population dynamics via a single partial differential equation (Theorem 1). Through a case study involving Cross learning agents, we illustrate that Theorem 1 allows us to identify qualitatively different evolutionary dynamics, to analyse steady states, and to gain insights into the expected behaviour of a population. In addition, we present extensive experimental results validating that Theorem 1 holds for a variety of learning methods

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhu2020the

Related papers

• Evolution Of Societies Via Reinforcement Learning (2024)0.00
• Game Theory And Multi-agent Reinforcement Learning : From Nash Equilibria To Evolutionary Dynamics (2024)0.00
• Independent Learning In Stochastic Games (2021)6.77
• Inclusive Fitness As A Key Step Towards More Advanced Social Behaviors In Multi-agent Reinforcement Learning Settings (2025)0.00
• The Dynamics Of Q-learning In Population Games: A Physics-inspired Continuity Equation Model (2022)0.00
• Learning Equilibrium With Estimated Payoffs In Population Games (2024)0.00
• Chaos Persists In Large-scale Multi-agent Learning Despite Adaptive Learning Rates (2023)0.00
• A Study Of AI Population Dynamics With Million-agent Reinforcement Learning (2017)0.00