MAESTRO: Open-ended Environment Design For Multi-agent Reinforcement Learning

Abstract

Open-ended learning methods that automatically generate a curriculum of increasingly challenging tasks serve as a promising avenue toward generally capable reinforcement learning agents. Existing methods adapt curricula independently over either environment parameters (in single-agent settings) or co-player policies (in multi-agent settings). However, the strengths and weaknesses of co-players can manifest themselves differently depending on environmental features. It is thus crucial to consider the dependency between the environment and co-player when shaping a curriculum in multi-agent domains. In this work, we use this insight and extend Unsupervised Environment Design (UED) to multi-agent environments. We then introduce Multi-Agent Environment Design Strategist for Open-Ended Learning (MAESTRO), the first multi-agent UED approach for two-player zero-sum settings. MAESTRO efficiently produces adversarial, joint curricula over both environments and co-players and attains minimax-regr

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysamvelyan2023maestro

Related papers

• Replay-guided Adversarial Environment Design (2021)0.00
• Emergent Complexity And Zero-shot Transfer Via Unsupervised Environment Design (2020)0.00
• Open-ended Learning Leads To Generally Capable Agents (2021)0.00
• Learning To Design Games: Strategic Environments In Reinforcement Learning (2017)0.00
• An Efficient Open World Environment For Multi-agent Social Learning (2025)0.00
• Emergence Of Collective Open-ended Exploration From Decentralized Meta-reinforcement Learning (2023)0.00
• Multi-agent Actor-critic For Mixed Cooperative-competitive Environments (2017)0.00
• Stackelberg Games For Learning Emergent Behaviors During Competitive Autocurricula (2023)5.84