Learning To Coordinate In Multi-agent Systems: A Coordinated Actor-critic Algorithm And Finite-time Guarantees

Abstract

Multi-agent reinforcement learning (MARL) has attracted much research attention recently. However, unlike its single-agent counterpart, many theoretical and algorithmic aspects of MARL have not been well-understood. In this paper, we study the emergence of coordinated behavior by autonomous agents using an actor-critic (AC) algorithm. Specifically, we propose and analyze a class of coordinated actor-critic algorithms (CAC) in which individually parametrized policies have a \{\it shared\} part (which is jointly optimized among all agents) and a \{\it personalized\} part (which is only locally optimized). Such kind of \{\it partially personalized\} policy allows agents to learn to coordinate by leveraging peers' past experience and adapt to individual tasks. The flexibility in our design allows the proposed MARL-CAC algorithm to be used in a \{\it fully decentralized\} setting, where the agents can only communicate with their neighbors, as well as a \{\it federated\} setting, where the a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzeng2021learning

Related papers

• Context-aware Bayesian Network Actor-critic Methods For Cooperative Multi-agent Reinforcement Learning (2023)0.00
• F2A2: Flexible Fully-decentralized Approximate Actor-critic For Cooperative Multi-agent Reinforcement Learning (2020)0.00
• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• Bi-level Actor-critic For Multi-agent Coordination (2019)0.00
• Learning To Advise And Learning From Advice In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00
• MARL With General Utilities Via Decentralized Shadow Reward Actor-critic (2021)0.00
• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00