Scalable Reinforcement Learning For Multi-agent Networked Systems

Abstract

We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized. A fundamental challenge in this setting is that the state-action space size scales exponentially in the number of agents, rendering the problem intractable for large networks. In this paper, we propose a Scalable Actor Critic (SAC) framework that exploits the network structure and finds a localized policy that is an \(O(\rho^\{\kappa\})\)-approximation of a stationary point of the objective for some \(\rho\in(0,1)\), with complexity that scales with the local state-action space size of the largest \(\kappa\)-hop neighborhood of the network. We illustrate our model and approach using examples from wireless communication, epidemics and traffic.

Stats

• citations23
• S2 citations—
• github stars0
• HF likes0
• heat score10.35
• arxiv keyqu2019scalable

Related papers

• Scalable Multi-agent Reinforcement Learning For Networked Systems With Average Reward (2020)0.00
• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00
• Causality Meets Locality: Provably Generalizable And Scalable Policy Learning For Networked Systems (2025)0.00
• Fully Decentralized Multi-agent Reinforcement Learning With Networked Agents (2018)0.00
• Scalable Centralized Deep Multi-agent Reinforcement Learning Via Policy Gradients (2018)0.00
• Global Convergence Of Localized Policy Iteration In Networked Multi-agent Reinforcement Learning (2022)2.26
• Scalable And Sample Efficient Distributed Policy Gradient Algorithms In Multi-agent Networked Systems (2022)0.00
• Transformer-based Scalable Multi-agent Reinforcement Learning For Networked Systems With Long-range Interactions (2025)0.00