HAVEN: Hierarchical Cooperative Multi-agent Reinforcement Learning With Dual Coordination Mechanism

Abstract

Recently, some challenging tasks in multi-agent systems have been solved by some hierarchical reinforcement learning methods. Inspired by the intra-level and inter-level coordination in the human nervous system, we propose a novel value decomposition framework HAVEN based on hierarchical reinforcement learning for fully cooperative multi-agent problems. To address the instability arising from the concurrent optimization of policies between various levels and agents, we introduce the dual coordination mechanism of inter-level and inter-agent strategies by designing reward functions in a two-level hierarchy. HAVEN does not require domain knowledge and pre-training, and can be applied to any value decomposition variant. Our method achieves desirable results on different decentralized partially observable Markov decision process domains and outperforms other popular multi-agent hierarchical reinforcement learning algorithms.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxu2021haven

Related papers

• Hierarchical Reinforcement Learning With Opponent Modeling For Distributed Multi-agent Cooperation (2022)5.84
• Feudal Multi-agent Hierarchies For Cooperative Reinforcement Learning (2019)0.00
• Strategic Coordination For Evolving Multi-agent Systems: A Hierarchical Reinforcement And Collective Learning Approach (2025)0.00
• Subgoal-based Hierarchical Reinforcement Learning For Multi-agent Collaboration (2024)0.00
• Hierarchical Reinforcement Learning For Optimal Agent Grouping In Cooperative Systems (2025)0.00
• Fully Decentralized Cooperative Multi-agent Reinforcement Learning: A Survey (2024)0.00
• TAG: A Decentralized Framework For Multi-agent Hierarchical Reinforcement Learning (2025)0.00
• Guiding Multi-agent Multi-task Reinforcement Learning By A Hierarchical Framework With Logical Reward Shaping (2024)0.00