F2A2: Flexible Fully-decentralized Approximate Actor-critic For Cooperative Multi-agent Reinforcement Learning

Abstract

Traditional centralized multi-agent reinforcement learning (MARL) algorithms are sometimes unpractical in complicated applications, due to non-interactivity between agents, curse of dimensionality and computation complexity. Hence, several decentralized MARL algorithms are motivated. However, existing decentralized methods only handle the fully cooperative setting where massive information needs to be transmitted in training. The block coordinate gradient descent scheme they used for successive independent actor and critic steps can simplify the calculation, but it causes serious bias. In this paper, we propose a flexible fully decentralized actor-critic MARL framework, which can combine most of actor-critic methods, and handle large-scale general cooperative multi-agent setting. A primal-dual hybrid gradient descent type algorithm framework is designed to learn individual agents separately for decentralization. From the perspective of each agent, policy improvement and value evaluatio

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2020f2a2

Related papers

• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• Learning To Coordinate In Multi-agent Systems: A Coordinated Actor-critic Algorithm And Finite-time Guarantees (2021)0.00
• Communication-efficient Actor-critic Methods For Homogeneous Markov Games (2022)0.00
• On Centralized Critics In Multi-agent Reinforcement Learning (2024)9.03
• Towards Global Optimality In Cooperative MARL With The Transformation And Distillation Framework (2022)0.00
• Contrasting Centralized And Decentralized Critics In Multi-agent Reinforcement Learning (2021)0.00
• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00
• MARL With General Utilities Via Decentralized Shadow Reward Actor-critic (2021)0.00