Learning And Exploiting Multiple Subgoals For Fast Exploration In Hierarchical Reinforcement Learning
2019 · Libo Xing
Abstract
Hierarchical Reinforcement Learning (HRL) exploits temporally extended actions, or options, to make decisions from a higher-dimensional perspective to alleviate the sparse reward problem, one of the most challenging problems in reinforcement learning. The majority of existing HRL algorithms require either significant manual design with respect to the specific environment or enormous exploration to automatically learn options from data. To achieve fast exploration without using manual design, we devise a multi-goal HRL algorithm, consisting of a high-level policy Manager and a low-level policy Worker. The Manager provides the Worker multiple subgoals at each time step. Each subgoal corresponds to an option to control the environment. Although the agent may show some confusion at the beginning of training since it is guided by three diverse subgoals, the agent's behavior policy will quickly learn how to respond to multiple subgoals from the high-level controller on different occasions. B
Authors
(none)
Tags
Stats
Related papers
- Learning Representations In Model-free Hierarchical Reinforcement Learning (2018)11.49
- Subgoal-based Hierarchical Reinforcement Learning For Multi-agent Collaboration (2024)0.00
- Efficient Exploration Through Intrinsic Motivation Learning For Unsupervised Subgoal Discovery In Model-free Hierarchical Reinforcement Learning (2019)0.00
- Generating Adjacency-constrained Subgoals In Hierarchical Reinforcement Learning (2020)0.00
- MENTOR: Guiding Hierarchical Reinforcement Learning With Human Feedback And Dynamic Distance Constraint (2024)6.34
- Exploring The Limits Of Hierarchical World Models In Reinforcement Learning (2024)6.34
- A Provably Efficient Option-based Algorithm For Both High-level And Low-level Learning (2024)0.00
- Boosting Hierarchical Reinforcement Learning With Meta-learning For Complex Task Adaptation (2024)0.00