Feudal Multi-agent Hierarchies For Cooperative Reinforcement Learning

Abstract

We investigate how reinforcement learning agents can learn to cooperate. Drawing inspiration from human societies, in which successful coordination of many individuals is often facilitated by hierarchical organisation, we introduce Feudal Multi-agent Hierarchies (FMH). In this framework, a 'manager' agent, which is tasked with maximising the environmentally-determined reward function, learns to communicate subgoals to multiple, simultaneously-operating, 'worker' agents. Workers, which are rewarded for achieving managerial subgoals, take concurrent actions in the world. We outline the structure of FMH and demonstrate its potential for decentralised learning and control. We find that, given an adequate set of subgoals from which to choose, FMH performs, and particularly scales, substantially better than cooperative approaches that use a shared reward function.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyahilan2019feudal

Related papers

• Feudal Networks For Hierarchical Reinforcement Learning (2017)0.00
• Subgoal-based Hierarchical Reinforcement Learning For Multi-agent Collaboration (2024)0.00
• HAVEN: Hierarchical Cooperative Multi-agent Reinforcement Learning With Dual Coordination Mechanism (2021)0.00
• Strategic Coordination For Evolving Multi-agent Systems: A Hierarchical Reinforcement And Collective Learning Approach (2025)0.00
• Fully Decentralized Cooperative Multi-agent Reinforcement Learning: A Survey (2024)0.00
• Guiding Multi-agent Multi-task Reinforcement Learning By A Hierarchical Framework With Logical Reward Shaping (2024)0.00
• Hierarchical Reinforcement Learning With Opponent Modeling For Distributed Multi-agent Cooperation (2022)5.84
• Modeling Sensorimotor Coordination As Multi-agent Reinforcement Learning With Differentiable Communication (2019)0.00