Sa-matd3:self-attention-based Multi-agent Continuous Control Method In Cooperative Environments

Abstract

Cooperative problems under continuous control have always been the focus of multi-agent reinforcement learning. Existing algorithms suffer from the problem of uneven learning degree with the increase of the number of agents. In this paper, a new structure for a multi-agent actor critic is proposed, and the self-attention mechanism is applied in the critic network and the value decomposition method used to solve the uneven problem. The proposed algorithm makes full use of the samples in the replay memory buffer to learn the behavior of a class of agents. First, a new update method is proposed for policy networks that promotes learning efficiency. Second, the utilization of samples is improved, at the same time reflecting the ability of perspective-taking among groups. Finally, the "deceptive signal" in training is eliminated and the learning degree among agents is more uniform than in the existing methods. Multiple experiments were conducted in two typical scenarios of a multi-agent par

Stats

• citations36
• S2 citations—
• github stars0
• HF likes0
• heat score11.76
• arxiv keyliu2021sa

Related papers

• Multi-agent Actor-critic For Mixed Cooperative-competitive Environments (2017)0.00
• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00
• Attraction-repulsion Actor-critic For Continuous Control Reinforcement Learning (2019)0.00
• Attention Actor-critic Algorithm For Multi-agent Constrained Co-operative Reinforcement Learning (2021)0.00
• Solving Continuous Control Via Q-learning (2022)0.00
• Decomposed Soft Actor-critic Method For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Bi-level Actor-critic For Multi-agent Coordination (2019)0.00
• A Multi-agent Off-policy Actor-critic Algorithm For Distributed Reinforcement Learning (2019)11.39