Convergence Of Heterogeneous Learning Dynamics In Zero-sum Stochastic Games

Abstract

This paper presents new families of algorithms for the repeated play of two-agent (near) zero-sum games and two-agent zero-sum stochastic games. For example, the family includes fictitious play and its variants as members. Commonly, the algorithms in this family are all uncoupled, rational, and convergent even in heterogeneous cases, e.g., where the dynamics may differ in terms of learning rates, full, none or temporal access to opponent actions, and model-based vs model-free learning. The convergence of heterogeneous dynamics is of practical interest especially in competitive environments since agents may have no means or interests in following the same dynamic with the same parameters. We prove that any mixture of such asymmetries does not impact the algorithms' convergence to equilibrium (or near equilibrium if there is experimentation) in zero-sum games with repeated play and in zero-sum (irreducible) stochastic games with sufficiently small discount factors.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyarslantas2023convergence

Related papers

• On The Heterogeneity Of Independent Learning Dynamics In Zero-sum Stochastic Games (2021)0.00
• Last-iterate Convergence Of Payoff-based Independent Learning In Zero-sum Stochastic Games (2024)0.00
• Best-response Dynamics And Fictitious Play In Identical-interest And Zero-sum Stochastic Games (2021)0.00
• Actor-dual-critic Dynamics For Zero-sum And Identical-interest Stochastic Games (2026)0.00
• Synchronization In Learning In Periodic Zero-sum Games Triggers Divergence From Nash Equilibrium (2024)0.00
• Convergence Analysis Of Gradient-based Learning With Non-uniform Learning Rates In Non-cooperative Multi-agent Settings (2019)0.00
• Memory Asymmetry Creates Heteroclinic Orbits To Nash Equilibrium In Learning In Zero-sum Games (2023)0.00
• A Finite-sample Analysis Of Payoff-based Independent Learning In Zero-sum Stochastic Games (2023)0.00