Scalable Neighborhood-based Multi-agent Actor-critic

Abstract

We propose MADDPG-K, a scalable extension to Multi-Agent Deep Deterministic Policy Gradient (MADDPG) that addresses the computational limitations of centralized critic approaches. Centralized critics, which condition on the observations and actions of all agents, have demonstrated significant performance gains in cooperative and competitive multi-agent settings. However, their critic networks grow linearly in input size with the number of agents, making them increasingly expensive to train at scale. MADDPG-K mitigates this by restricting each agent's critic to the \(k\) closest agents under a chosen metric which in our case is Euclidean distance. This ensures a constant-size critic input regardless of the total agent count. We analyze the complexity of this approach, showing that the quadratic cost it retains arises from cheap scalar distance computations rather than the expensive neural network matrix multiplications that bottleneck standard MADDPG. We validate our method empirically

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygoppelsroeder2026scalable

Related papers

• 3DPG: Distributed Deep Deterministic Policy Gradient Algorithms For Networked Multi-agent Systems (2022)0.00
• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00
• Contrasting Centralized And Decentralized Critics In Multi-agent Reinforcement Learning (2021)0.00
• Local Advantage Actor-critic For Robust Multi-agent Deep Reinforcement Learning (2021)7.81
• FACMAC: Factored Multi-agent Centralised Policy Gradients (2020)0.00
• Parameter Sharing Deep Deterministic Policy Gradient For Cooperative Multi-agent Reinforcement Learning (2017)0.00
• Multi-agent Actor-critic For Mixed Cooperative-competitive Environments (2017)0.00
• F2A2: Flexible Fully-decentralized Approximate Actor-critic For Cooperative Multi-agent Reinforcement Learning (2020)0.00