Heterogeneous Multi-agent Reinforcement Learning For Zero-shot Scalable Collaboration

Abstract

The emergence of multi-agent reinforcement learning (MARL) is significantly transforming various fields like autonomous vehicle networks. However, real-world multi-agent systems typically contain multiple roles, and the scale of these systems dynamically fluctuates. Consequently, in order to achieve zero-shot scalable collaboration, it is essential that strategies for different roles can be updated flexibly according to the scales, which is still a challenge for current MARL frameworks. To address this, we propose a novel MARL framework named Scalable and Heterogeneous Proximal Policy Optimization (SHPPO), integrating heterogeneity into parameter-shared PPO-based MARL networks. We first leverage a latent network to learn strategy patterns for each agent adaptively. Second, we introduce a heterogeneous layer to be inserted into decision-making networks, whose parameters are specifically generated by the learned latent variables. Our approach is scalable as all the parameters are shared

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keyguo2024heterogeneous

Related papers

• Heterogeneous Multi-robot Reinforcement Learning (2023)6.77
• Heterogeneous-agent Reinforcement Learning (2023)0.00
• Hypermarl: Adaptive Hypernetworks For Multi-agent RL (2024)0.00
• Role Play: Learning Adaptive Role-specific Strategies In Multi-agent Interactions (2024)0.00
• End-to-end Optimization Of Llm-driven Multi-agent Search Systems Via Heterogeneous-group-based Reinforcement Learning (2025)0.00
• Maximum Entropy Heterogeneous-agent Reinforcement Learning (2023)0.00
• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00
• Cooperative Multi-agent Reinforcement Learning With Partial Observations (2020)10.35