Transformer-based Value Function Decomposition For Cooperative Multi-agent Reinforcement Learning In Starcraft

Abstract

The StarCraft II Multi-Agent Challenge (SMAC) was created to be a challenging benchmark problem for cooperative multi-agent reinforcement learning (MARL). SMAC focuses exclusively on the problem of StarCraft micromanagement and assumes that each unit is controlled individually by a learning agent that acts independently and only possesses local information; centralized training is assumed to occur with decentralized execution (CTDE). To perform well in SMAC, MARL algorithms must handle the dual problems of multi-agent credit assignment and joint action evaluation. This paper introduces a new architecture TransMix, a transformer-based joint action-value mixing network which we show to be efficient and scalable as compared to the other state-of-the-art cooperative MARL solutions. TransMix leverages the ability of transformers to learn a richer mixing function for combining the agents' individual value functions. It achieves comparable performance to previous work on easy SMAC scenarios

Stats

• citations14
• S2 citations—
• github stars0
• HF likes0
• heat score8.82
• arxiv keykhan2022transformer

Related papers

• Smacv2: An Improved Benchmark For Cooperative Multi-agent Reinforcement Learning (2022)5.24
• Decomposed Soft Actor-critic Method For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Cooperative Multi-agent Reinforcement Learning With Hypergraph Convolution (2021)5.84
• Starcraft+: Benchmarking Multi-agent Algorithms In Adversary Paradigm (2025)0.00
• Smac-hard: Enabling Mixed Opponent Strategy Script And Self-play On SMAC (2024)0.00
• Transformer-based Scalable Multi-agent Reinforcement Learning For Networked Systems With Long-range Interactions (2025)0.00
• Is Independent Learning All You Need In The Starcraft Multi-agent Challenge? (2020)0.00
• Modeling The Interaction Between Agents In Cooperative Multi-agent Reinforcement Learning (2021)0.00