Scalable Multi-agent Reinforcement Learning For Networked Systems With Average Reward

Abstract

It has long been recognized that multi-agent reinforcement learning (MARL) faces significant scalability issues due to the fact that the size of the state and action spaces are exponentially large in the number of agents. In this paper, we identify a rich class of networked MARL problems where the model exhibits a local dependence structure that allows it to be solved in a scalable manner. Specifically, we propose a Scalable Actor-Critic (SAC) method that can learn a near optimal localized policy for optimizing the average reward with complexity scaling with the state-action space size of local neighborhoods, as opposed to the entire network. Our result centers around identifying and exploiting an exponential decay property that ensures the effect of agents on each other decays exponentially fast in their graph distance.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyqu2020scalable

Related papers

• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00
• Scalable Reinforcement Learning For Multi-agent Networked Systems (2019)10.35
• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• Transformer-based Scalable Multi-agent Reinforcement Learning For Networked Systems With Long-range Interactions (2025)0.00
• Scalable And Sample Efficient Distributed Policy Gradient Algorithms In Multi-agent Networked Systems (2022)0.00
• Locality Matters: A Scalable Value Decomposition Approach For Cooperative Multi-agent Reinforcement Learning (2021)0.00
• Mean-field Multi-agent Reinforcement Learning: A Decentralized Network Approach (2021)0.00
• Local Advantage Networks For Cooperative Multi-agent Reinforcement Learning (2021)0.00