Developing Cooperative Policies For Multi-stage Reinforcement Learning Tasks

Abstract

Many hierarchical reinforcement learning algorithms utilise a series of independent skills as a basis to solve tasks at a higher level of reasoning. These algorithms don't consider the value of using skills that are cooperative instead of independent. This paper proposes the Cooperative Consecutive Policies (CCP) method of enabling consecutive agents to cooperatively solve long time horizon multi-stage tasks. This method is achieved by modifying the policy of each agent to maximise both the current and next agent's critic. Cooperatively maximising critics allows each agent to take actions that are beneficial for its task as well as subsequent tasks. Using this method in a multi-room maze domain and a peg in hole manipulation domain, the cooperative policies were able to outperform a set of naive policies, a single agent trained across the entire domain, as well as another sequential HRL algorithm.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyerskine2022developing

Related papers

• Hierarchical And Interpretable Skill Acquisition In Multi-task Reinforcement Learning (2017)0.00
• Bidirectional-reachable Hierarchical Reinforcement Learning With Mutually Responsive Policies (2024)0.00
• Multi-agent Actor-critic For Mixed Cooperative-competitive Environments (2017)0.00
• Multi-agent Common Knowledge Reinforcement Learning (2018)0.00
• Hierarchical Reinforcement Learning For Optimal Agent Grouping In Cooperative Systems (2025)0.00
• Hierarchical Reinforcement Learning With Opponent Modeling For Distributed Multi-agent Cooperation (2022)5.84
• Hierarchical Reinforcement Learning With Advantage-based Auxiliary Rewards (2019)0.00
• Behaviour-conditioned Policies For Cooperative Reinforcement Learning Tasks (2021)2.26