Scalable Centralized Deep Multi-agent Reinforcement Learning Via Policy Gradients

Abstract

In this paper, we explore using deep reinforcement learning for problems with multiple agents. Most existing methods for deep multi-agent reinforcement learning consider only a small number of agents. When the number of agents increases, the dimensionality of the input and control spaces increase as well, and these methods do not scale well. To address this, we propose casting the multi-agent reinforcement learning problem as a distributed optimization problem. Our algorithm assumes that for multi-agent settings, policies of individual agents in a given population live close to each other in parameter space and can be approximated by a single policy. With this simple assumption, we show our algorithm to be extremely effective for reinforcement learning in multi-agent settings. We demonstrate its effectiveness against existing comparable approaches on co-operative and competitive tasks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykhan2018scalable

Related papers

• Parameter Sharing Deep Deterministic Policy Gradient For Cooperative Multi-agent Reinforcement Learning (2017)0.00
• Scalable And Sample Efficient Distributed Policy Gradient Algorithms In Multi-agent Networked Systems (2022)0.00
• Communication-efficient Policy Gradient Methods For Distributed Reinforcement Learning (2018)13.05
• Deep Multiagent Reinforcement Learning: Challenges And Directions (2021)0.00
• Dimension-free Rates For Natural Policy Gradient In Multi-agent Reinforcement Learning (2021)0.00
• A Policy Gradient Algorithm For Learning To Learn In Multiagent Reinforcement Learning (2020)0.00
• A New Framework For Multi-agent Reinforcement Learning -- Centralized Training And Exploration With Decentralized Execution Via Policy Distillation (2019)0.00
• Descent-guided Policy Gradient For Scalable Cooperative Multi-agent Learning (2026)0.00