Attention-based Recurrence For Multi-agent Reinforcement Learning Under Stochastic Partial Observability

Abstract

Stochastic partial observability poses a major challenge for decentralized coordination in multi-agent reinforcement learning but is largely neglected in state-of-the-art research due to a strong focus on state-based centralized training for decentralized execution (CTDE) and benchmarks that lack sufficient stochasticity like StarCraft Multi-Agent Challenge (SMAC). In this paper, we propose Attention-based Embeddings of Recurrence In multi-Agent Learning (AERIAL) to approximate value functions under stochastic partial observability. AERIAL replaces the true state with a learned representation of multi-agent recurrence, considering more accurate information about decentralized agent decisions than state-based CTDE. We then introduce MessySMAC, a modified version of SMAC with stochastic observations and higher variance in initial states, to provide a more general and configurable benchmark regarding stochastic partial observability. We evaluate AERIAL in Dec-Tiger as well as in a variety

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyphan2023attention

Related papers

• Deep Decentralized Multi-task Multi-agent Reinforcement Learning Under Partial Observability (2017)0.00
• A Decentralized Communication Framework Based On Dual-level Recurrence For Multi-agent Reinforcement Learning (2022)5.84
• Smacv2: An Improved Benchmark For Cooperative Multi-agent Reinforcement Learning (2022)5.24
• DSDF: An Approach To Handle Stochastic Agents In Collaborative Multi-agent Reinforcement Learning (2021)0.00
• Agent-time Attention For Sparse Rewards Multi-agent Reinforcement Learning (2022)0.00
• STAS: Spatial-temporal Return Decomposition For Multi-agent Reinforcement Learning (2023)0.00
• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00
• Attention-guided Contrastive Role Representations For Multi-agent Reinforcement Learning (2023)3.64