Accelerate Multi-agent Reinforcement Learning In Zero-sum Games With Subgame Curriculum Learning

Abstract

Learning Nash equilibrium (NE) in complex zero-sum games with multi-agent reinforcement learning (MARL) can be extremely computationally expensive. Curriculum learning is an effective way to accelerate learning, but an under-explored dimension for generating a curriculum is the difficulty-to-learn of the subgames -- games induced by starting from a specific state. In this work, we present a novel subgame curriculum learning framework for zero-sum games. It adopts an adaptive initial state distribution by resetting agents to some previously visited states where they can quickly learn to improve performance. Building upon this framework, we derive a subgame selection metric that approximates the squared distance to NE values and further adopt a particle-based state sampler for subgame generation. Integrating these techniques leads to our new algorithm, Subgame Automatic Curriculum Learning (SACL), which is a realization of the subgame curriculum learning framework. SACL can be combined w

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2023accelerate

Related papers

• Neural Auto-curricula (2021)0.00
• Towards Skilled Population Curriculum For Multi-agent Reinforcement Learning (2023)0.00
• Curriculum Learning With Counterfactual Group Relative Policy Advantage For Multi-agent Reinforcement Learning (2025)0.00
• Decentralized Q-learning In Zero-sum Markov Games (2021)0.00
• Learning Progress Driven Multi-agent Curriculum (2022)0.00
• Stackelberg Games For Learning Emergent Behaviors During Competitive Autocurricula (2023)5.84
• Colosseumrl: A Framework For Multiagent Reinforcement Learning In \(n\)-player Games (2019)0.00
• Variational Automatic Curriculum Learning For Sparse-reward Cooperative Multi-agent Problems (2021)0.00