Kaleidoscope: Learnable Masks For Heterogeneous Multi-agent Reinforcement Learning

Abstract

In multi-agent reinforcement learning (MARL), parameter sharing is commonly employed to enhance sample efficiency. However, the popular approach of full parameter sharing often leads to homogeneous policies among agents, potentially limiting the performance benefits that could be derived from policy diversity. To address this critical limitation, we introduce *Kaleidoscope*, a novel adaptive partial parameter sharing scheme that fosters policy heterogeneity while still maintaining high sample efficiency. Specifically, Kaleidoscope maintains one set of common parameters alongside multiple sets of distinct, learnable masks for different agents, dictating the sharing of parameters. It promotes diversity among policy networks by encouraging discrepancy among these masks, without sacrificing the efficiencies of parameter sharing. This design allows Kaleidoscope to dynamically balance high sample efficiency with a broad policy representational capacity, effectively bridging the gap between f

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyli2024kaleidoscope

Related papers

• Adaptive Parameter Sharing For Multi-agent Reinforcement Learning (2023)0.00
• Hypermarl: Adaptive Hypernetworks For Multi-agent RL (2024)0.00
• Revisiting Parameter Sharing In Multi-agent Deep Reinforcement Learning (2020)0.00
• Scaling Multi-agent Reinforcement Learning With Selective Parameter Sharing (2021)0.00
• Heterogeneous Multi-agent Reinforcement Learning For Zero-shot Scalable Collaboration (2024)6.34
• Parameter Sharing With Network Pruning For Scalable Multi-agent Deep Reinforcement Learning (2023)2.26
• Improving Global Parameter-sharing In Physically Heterogeneous Multi-agent Reinforcement Learning With Unified Action Space (2024)0.00
• Revisiting Some Common Practices In Cooperative Multi-agent Reinforcement Learning (2022)0.00