Stackelberg Games For Learning Emergent Behaviors During Competitive Autocurricula

Abstract

Autocurricular training is an important sub-area of multi-agent reinforcement learning~(MARL) that allows multiple agents to learn emergent skills in an unsupervised co-evolving scheme. The robotics community has experimented autocurricular training with physically grounded problems, such as robust control and interactive manipulation tasks. However, the asymmetric nature of these tasks makes the generation of sophisticated policies challenging. Indeed, the asymmetry in the environment may implicitly or explicitly provide an advantage to a subset of agents which could, in turn, lead to a low-quality equilibrium. This paper proposes a novel game-theoretic algorithm, Stackelberg Multi-Agent Deep Deterministic Policy Gradient (ST-MADDPG), which formulates a two-player MARL problem as a Stackelberg game with one player as the `leader' and the other as the `follower' in a hierarchical interaction structure wherein the leader has an advantage. We first demonstrate that the leader's advantage

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyyang2023stackelberg

Related papers

• Neural Auto-curricula (2021)0.00
• Imitation Learning Of Correlated Policies In Stackelberg Games (2025)0.00
• Inducing Stackelberg Equilibrium Through Spatio-temporal Sequential Decision-making In Multi-agent Reinforcement Learning (2023)7.50
• Curriculum Learning With Counterfactual Group Relative Policy Advantage For Multi-agent Reinforcement Learning (2025)0.00
• Oracles & Followers: Stackelberg Equilibria In Deep Multi-agent Reinforcement Learning (2022)0.00
• Towards Skilled Population Curriculum For Multi-agent Reinforcement Learning (2023)0.00
• Evolutionary Population Curriculum For Scaling Multi-agent Reinforcement Learning (2020)0.00
• Autotelic Reinforcement Learning In Multi-agent Environments (2022)0.00