Hierarchical Reinforcement Learning In Complex 3D Environments

Abstract

Hierarchical Reinforcement Learning (HRL) agents have the potential to demonstrate appealing capabilities such as planning and exploration with abstraction, transfer, and skill reuse. Recent successes with HRL across different domains provide evidence that practical, effective HRL agents are possible, even if existing agents do not yet fully realize the potential of HRL. Despite these successes, visually complex partially observable 3D environments remained a challenge for HRL agents. We address this issue with Hierarchical Hybrid Offline-Online (H2O2), a hierarchical deep reinforcement learning agent that discovers and learns to use options from scratch using its own experience. We show that H2O2 is competitive with a strong non-hierarchical Muesli baseline in the DeepMind Hard Eight tasks and we shed new light on the problem of learning hierarchical agents in complex environments. Our empirical study of H2O2 reveals previously unnoticed practical challenges and brings new perspective

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypires2023hierarchical

Related papers

• Exploring The Limits Of Hierarchical World Models In Reinforcement Learning (2024)6.34
• Learning How To Interact With A Complex Interface Using Hierarchical Reinforcement Learning (2022)0.00
• H2O+: An Improved Framework For Hybrid Offline-and-online RL With Dynamics Gaps (2023)0.00
• Learning And Exploiting Multiple Subgoals For Fast Exploration In Hierarchical Reinforcement Learning (2019)0.00
• Boosting Hierarchical Reinforcement Learning With Meta-learning For Complex Task Adaptation (2024)0.00
• Hierarchical Decision Making Based On Structural Information Principles (2024)0.00
• MENTOR: Guiding Hierarchical Reinforcement Learning With Human Feedback And Dynamic Distance Constraint (2024)6.34
• Subgoal-based Hierarchical Reinforcement Learning For Multi-agent Collaboration (2024)0.00