Variational Automatic Curriculum Learning For Sparse-reward Cooperative Multi-agent Problems

Abstract

We introduce a curriculum learning algorithm, Variational Automatic Curriculum Learning (VACL), for solving challenging goal-conditioned cooperative multi-agent reinforcement learning problems. We motivate our paradigm through a variational perspective, where the learning objective can be decomposed into two terms: task learning on the current task distribution, and curriculum update to a new task distribution. Local optimization over the second term suggests that the curriculum should gradually expand the training tasks from easy to hard. Our VACL algorithm implements this variational paradigm with two practical components, task expansion and entity progression, which produces training curricula over both the task configurations as well as the number of entities in the task. Experiment results show that VACL solves a collection of sparse-reward problems with a large number of agents. Particularly, using a single desktop machine, VACL achieves 98% coverage rate with 100 agents in the s

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2021variational

Related papers

• Towards Skilled Population Curriculum For Multi-agent Reinforcement Learning (2023)0.00
• Curriculum Learning For Cooperation In Multi-agent Reinforcement Learning (2023)0.00
• Learning Progress Driven Multi-agent Curriculum (2022)0.00
• V-learning -- A Simple, Efficient, Decentralized Algorithm For Multiagent RL (2021)0.00
• Curriculum Learning With Counterfactual Group Relative Policy Advantage For Multi-agent Reinforcement Learning (2025)0.00
• Accelerate Multi-agent Reinforcement Learning In Zero-sum Games With Subgame Curriculum Learning (2023)0.00
• Stein Variational Goal Generation For Adaptive Exploration In Multi-goal Reinforcement Learning (2022)0.00
• Variational Policy Propagation For Multi-agent Reinforcement Learning (2020)0.00