Distributed Multi-agent Reinforcement Learning Based On Graph-induced Local Value Functions

Abstract

Achieving distributed reinforcement learning (RL) for large-scale cooperative multi-agent systems (MASs) is challenging because: (i) each agent has access to only limited information; (ii) issues on convergence or computational complexity emerge due to the curse of dimensionality. In this paper, we propose a general computationally efficient distributed framework for cooperative multi-agent reinforcement learning (MARL) by utilizing the structures of graphs involved in this problem. We introduce three coupling graphs describing three types of inter-agent couplings in MARL, namely, the state graph, the observation graph and the reward graph. By further considering a communication graph, we propose two distributed RL approaches based on local value-functions derived from the coupling graphs. The first approach is able to reduce sample complexity significantly under specific conditions on the aforementioned four graphs. The second approach provides an approximate solution and can be effic

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyjing2022distributed

Related papers

• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• Locality Matters: A Scalable Value Decomposition Approach For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Mean-field Multi-agent Reinforcement Learning: A Decentralized Network Approach (2021)0.00
• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00
• Decentralized Multi-agent Reinforcement Learning With Networked Agents: Recent Advances (2019)0.00
• Factored Value Functions For Graph-based Multi-agent Reinforcement Learning (2026)0.00
• Group-aware Coordination Graph For Multi-agent Reinforcement Learning (2024)0.00
• Scalable Multi-agent Reinforcement Learning For Networked Systems With Average Reward (2020)0.00