Hierarchical Meta-reinforcement Learning Via Automated Macro-action Discovery

Abstract

Meta-Reinforcement Learning (Meta-RL) enables fast adaptation to new testing tasks. Despite recent advancements, it is still challenging to learn performant policies across multiple complex and high-dimensional tasks. To address this, we propose a novel architecture with three hierarchical levels for 1) learning task representations, 2) discovering task-agnostic macro-actions in an automated manner, and 3) learning primitive actions. The macro-action can guide the low-level primitive policy learning to more efficiently transition to goal states. This can address the issue that the policy may forget previously learned behavior while learning new, conflicting tasks. Moreover, the task-agnostic nature of the macro-actions is enabled by removing task-specific components from the state space. Hence, this makes them amenable to re-composition across different tasks and leads to promising fast adaptation to new tasks. Also, the prospective instability from the tri-level hierarchies is effecti

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycho2024hierarchical

Related papers

• Boosting Hierarchical Reinforcement Learning With Meta-learning For Complex Task Adaptation (2024)0.00
• Hypothesis Network Planned Exploration For Rapid Meta-reinforcement Learning Adaptation (2023)0.00
• Reusability And Transferability Of Macro Actions For Reinforcement Learning (2019)0.00
• Guided Meta-policy Search (2019)0.00
• Context Meta-reinforcement Learning Via Neuromodulation (2021)6.34
• A Tutorial On Meta-reinforcement Learning (2023)10.85
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Decoupling Exploration And Exploitation For Meta-reinforcement Learning Without Sacrifices (2020)0.00