Improving Global Parameter-sharing In Physically Heterogeneous Multi-agent Reinforcement Learning With Unified Action Space

Abstract

In a multi-agent system (MAS), action semantics indicates the different influences of agents' actions toward other entities, and can be used to divide agents into groups in a physically heterogeneous MAS. Previous multi-agent reinforcement learning (MARL) algorithms apply global parameter-sharing across different types of heterogeneous agents without careful discrimination of different action semantics. This common implementation decreases the cooperation and coordination between agents in complex situations. However, fully independent agent parameters dramatically increase the computational cost and training difficulty. In order to benefit from the usage of different action semantics while also maintaining a proper parameter-sharing structure, we introduce the Unified Action Space (UAS) to fulfill the requirement. The UAS is the union set of all agent actions with different semantics. All agents first calculate their unified representation in the UAS, and then generate their heterogen

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyu2024improving

Related papers

• Adaptive Parameter Sharing For Multi-agent Reinforcement Learning (2023)0.00
• Revisiting Parameter Sharing In Multi-agent Deep Reinforcement Learning (2020)0.00
• Scaling Multi-agent Reinforcement Learning With Selective Parameter Sharing (2021)0.00
• Hypermarl: Adaptive Hypernetworks For Multi-agent RL (2024)0.00
• Kaleidoscope: Learnable Masks For Heterogeneous Multi-agent Reinforcement Learning (2024)2.26
• A Further Exploration Of Deep Multi-agent Reinforcement Learning With Hybrid Action Space (2022)5.84
• Deep Multi-agent Reinforcement Learning With Hybrid Action Spaces Based On Maximum Entropy (2022)0.00
• Parameter Sharing With Network Pruning For Scalable Multi-agent Deep Reinforcement Learning (2023)2.26