Learning Progress Driven Multi-agent Curriculum

Abstract

The number of agents can be an effective curriculum variable for controlling the difficulty of multi-agent reinforcement learning (MARL) tasks. Existing work typically uses manually defined curricula such as linear schemes. We identify two potential flaws while applying existing reward-based automatic curriculum learning methods in MARL: (1) The expected episode return used to measure task difficulty has high variance; (2) Credit assignment difficulty can be exacerbated in tasks where increasing the number of agents yields higher returns which is common in many MARL tasks. To address these issues, we propose to control the curriculum by using a TD-error based *learning progress* measure and by letting the curriculum proceed from an initial context distribution to the final task specific one. Since our approach maintains a distribution over the number of agents and measures learning progress rather than absolute performance, which often increases with the number of agents, we alleviate

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhao2022learning

Related papers

• Towards Skilled Population Curriculum For Multi-agent Reinforcement Learning (2023)0.00
• Curriculum Learning With Counterfactual Group Relative Policy Advantage For Multi-agent Reinforcement Learning (2025)0.00
• Curriculum Learning With A Progression Function (2020)0.00
• Evolutionary Population Curriculum For Scaling Multi-agent Reinforcement Learning (2020)0.00
• Learning Curriculum Policies For Reinforcement Learning (2018)5.24
• Curriculum Learning For Cooperation In Multi-agent Reinforcement Learning (2023)0.00
• Robustness To Multi-modal Environment Uncertainty In MARL Using Curriculum Learning (2023)0.00
• Variational Automatic Curriculum Learning For Sparse-reward Cooperative Multi-agent Problems (2021)0.00