Parameter Sharing With Network Pruning For Scalable Multi-agent Deep Reinforcement Learning

Abstract

Handling the problem of scalability is one of the essential issues for multi-agent reinforcement learning (MARL) algorithms to be applied to real-world problems typically involving massively many agents. For this, parameter sharing across multiple agents has widely been used since it reduces the training time by decreasing the number of parameters and increasing the sample efficiency. However, using the same parameters across agents limits the representational capacity of the joint policy and consequently, the performance can be degraded in multi-agent tasks that require different behaviors for different agents. In this paper, we propose a simple method that adopts structured pruning for a deep neural network to increase the representational capacity of the joint policy without introducing additional parameters. We evaluate the proposed method on several benchmark tasks, and numerical results show that the proposed method significantly outperforms other parameter-sharing methods.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keykim2023parameter

Related papers

• Scaling Multi-agent Reinforcement Learning With Selective Parameter Sharing (2021)0.00
• Adaptive Parameter Sharing For Multi-agent Reinforcement Learning (2023)0.00
• Parameter Sharing Deep Deterministic Policy Gradient For Cooperative Multi-agent Reinforcement Learning (2017)0.00
• Revisiting Parameter Sharing In Multi-agent Deep Reinforcement Learning (2020)0.00
• Hypermarl: Adaptive Hypernetworks For Multi-agent RL (2024)0.00
• Revisiting Some Common Practices In Cooperative Multi-agent Reinforcement Learning (2022)0.00
• Kaleidoscope: Learnable Masks For Heterogeneous Multi-agent Reinforcement Learning (2024)2.26
• PPS-QMIX: Periodically Parameter Sharing For Accelerating Convergence Of Multi-agent Reinforcement Learning (2024)1.81