Distributed Multi-agent Reinforcement Learning With One-hop Neighbors And Compute Straggler Mitigation

Abstract

Most multi-agent reinforcement learning (MARL) methods are limited in the scale of problems they can handle. With increasing numbers of agents, the number of training iterations required to find the optimal behaviors increases exponentially due to the exponentially growing joint state and action spaces. This paper tackles this limitation by introducing a scalable MARL method called Distributed multi-Agent Reinforcement Learning with One-hop Neighbors (DARL1N). DARL1N is an off-policy actor-critic method that addresses the curse of dimensionality by restricting information exchanges among the agents to one-hop neighbors when representing value and policy functions. Each agent optimizes its value and policy functions over a one-hop neighborhood, significantly reducing the learning complexity, yet maintaining expressiveness by training with varying neighbor numbers and states. This structure allows us to formulate a distributed learning framework to further speed up the training procedure

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2022distributed

Related papers

• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00
• Scalable Multi-agent Reinforcement Learning For Networked Systems With Average Reward (2020)0.00
• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• MARL-LNS: Cooperative Multi-agent Reinforcement Learning Via Large Neighborhoods Search (2024)0.00
• Cooperative Multi-agent Reinforcement Learning With Partial Observations (2020)10.35
• Scalable And Sample Efficient Distributed Policy Gradient Algorithms In Multi-agent Networked Systems (2022)0.00
• Local Advantage Networks For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Mean-field Multi-agent Reinforcement Learning: A Decentralized Network Approach (2021)0.00