Learning Complex Teamwork Tasks Using A Given Sub-task Decomposition

Abstract

Training a team to complete a complex task via multi-agent reinforcement learning can be difficult due to challenges such as policy search in a large joint policy space, and non-stationarity caused by mutually adapting agents. To facilitate efficient learning of complex multi-agent tasks, we propose an approach which uses an expert-provided decomposition of a task into simpler multi-agent sub-tasks. In each sub-task, a subset of the entire team is trained to acquire sub-task-specific policies. The sub-teams are then merged and transferred to the target task, where their policies are collectively fine-tuned to solve the more complex target task. We show empirically that such approaches can greatly reduce the number of timesteps required to solve a complex target task relative to training from-scratch. However, we also identify and investigate two problems with naive implementations of approaches based on sub-task decomposition, and propose a simple and scalable method to address these p

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfosong2023learning

Related papers

• ALMA: Hierarchical Learning For Composite Multi-agent Tasks (2022)0.00
• Deep Decentralized Multi-task Multi-agent Reinforcement Learning Under Partial Observability (2017)0.00
• Separation Of Concerns In Reinforcement Learning (2016)0.00
• LDSA: Learning Dynamic Subtask Assignment In Cooperative Multi-agent Reinforcement Learning (2022)3.58
• RODE: Learning Roles To Decompose Multi-agent Tasks (2020)0.00
• State Space Decomposition And Subgoal Creation For Transfer In Deep Reinforcement Learning (2017)0.00
• Hierarchical And Interpretable Skill Acquisition In Multi-task Reinforcement Learning (2017)0.00
• Policy Distillation And Value Matching In Multiagent Reinforcement Learning (2019)10.48