Is Independent Learning All You Need In The Starcraft Multi-agent Challenge?

Abstract

Most recently developed approaches to cooperative multi-agent reinforcement learning in the *centralized training with decentralized execution* setting involve estimating a centralized, joint value function. In this paper, we demonstrate that, despite its various theoretical shortcomings, Independent PPO (IPPO), a form of independent learning in which each agent simply estimates its local value function, can perform just as well as or better than state-of-the-art joint learning approaches on popular multi-agent benchmark suite SMAC with little hyperparameter tuning. We also compare IPPO to several variants; the results suggest that IPPO's strong performance may be due to its robustness to some forms of environment non-stationarity.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydewitt2020is

Related papers

• Transformer-based Value Function Decomposition For Cooperative Multi-agent Reinforcement Learning In Starcraft (2022)8.82
• Local Advantage Networks For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Jointppo: Diving Deeper Into The Effectiveness Of PPO In Multi-agent Reinforcement Learning (2024)0.00
• Smacv2: An Improved Benchmark For Cooperative Multi-agent Reinforcement Learning (2022)5.24
• Decentralized Multi-agents By Imitation Of A Centralized Controller (2019)0.00
• Fully Decentralized Cooperative Multi-agent Reinforcement Learning: A Survey (2024)0.00
• From Centralized To Self-supervised: Pursuing Realistic Multi-agent Reinforcement Learning (2023)0.00
• Investigation Of Independent Reinforcement Learning Algorithms In Multi-agent Environments (2021)0.00