Heterogeneous-agent Mirror Learning: A Continuum Of Solutions To Cooperative MARL

Abstract

The necessity for cooperation among intelligent machines has popularised cooperative multi-agent reinforcement learning (MARL) in the artificial intelligence (AI) research community. However, many research endeavors have been focused on developing practical MARL algorithms whose effectiveness has been studied only empirically, thereby lacking theoretical guarantees. As recent studies have revealed, MARL methods often achieve performance that is unstable in terms of reward monotonicity or suboptimal at convergence. To resolve these issues, in this paper, we introduce a novel framework named Heterogeneous-Agent Mirror Learning (HAML) that provides a general template for MARL algorithmic designs. We prove that algorithms derived from the HAML template satisfy the desired properties of the monotonic improvement of the joint reward and the convergence to Nash equilibrium. We verify the practicality of HAML by proving that the current state-of-the-art cooperative MARL algorithms, HATRPO and

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykuba2022heterogeneous

Related papers

• Heterogeneous-agent Reinforcement Learning (2023)0.00
• Heterogeneous Multi-agent Reinforcement Learning Via Mirror Descent Policy Optimization (2023)0.00
• Maximum Entropy Heterogeneous-agent Reinforcement Learning (2023)0.00
• Enhancing Heterogeneous Multi-agent Cooperation In Decentralized MARL Via Gnn-driven Intrinsic Rewards (2024)0.00
• Heterogeneous Multi-agent Reinforcement Learning For Zero-shot Scalable Collaboration (2024)6.34
• GHQ: Grouped Hybrid Q Learning For Heterogeneous Cooperative Multi-agent Reinforcement Learning (2023)6.34
• Hypermarl: Adaptive Hypernetworks For Multi-agent RL (2024)0.00
• A Review Of Cooperative Multi-agent Deep Reinforcement Learning (2019)19.08