Benchmarking Multi-agent Deep Reinforcement Learning Algorithms In Cooperative Tasks

Abstract

Multi-agent deep reinforcement learning (MARL) suffers from a lack of commonly-used evaluation tasks and criteria, making comparisons between approaches difficult. In this work, we provide a systematic evaluation and comparison of three different classes of MARL algorithms (independent learning, centralised multi-agent policy gradient, value decomposition) in a diverse range of cooperative multi-agent learning tasks. Our experiments serve as a reference for the expected performance of algorithms across different learning tasks, and we provide insights regarding the effectiveness of different learning approaches. We open-source EPyMARL, which extends the PyMARL codebase to include additional algorithms and allow for flexible configuration of algorithm implementation details such as parameter sharing. Finally, we open-source two environments for multi-agent research which focus on coordination under sparse rewards.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypapoudakis2020benchmarking

Related papers

• Towards A Standardised Performance Evaluation Protocol For Cooperative MARL (2022)0.00
• A Review Of Cooperative Multi-agent Deep Reinforcement Learning (2019)19.08
• How Much Can Change In A Year? Revisiting Evaluation In Multi-agent Reinforcement Learning (2023)0.00
• Revisiting Some Common Practices In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Policy Distillation And Value Matching In Multiagent Reinforcement Learning (2019)10.48
• Benchmarl: Benchmarking Multi-agent Reinforcement Learning (2023)5.58
• Marllib: A Scalable And Efficient Multi-agent Reinforcement Learning Library (2022)0.00
• An Initial Introduction To Cooperative Multi-agent Reinforcement Learning (2024)0.00