Learning Representations In Model-free Hierarchical Reinforcement Learning
2018 · Jacob Rafati, David C. Noelle
Abstract
Common approaches to Reinforcement Learning (RL) are seriously challenged by large-scale applications involving huge state spaces and sparse delayed reward feedback. Hierarchical Reinforcement Learning (HRL) methods attempt to address this scalability issue by learning action selection policies at multiple levels of temporal abstraction. Abstraction can be had by identifying a relatively small set of states that are likely to be useful as subgoals, in concert with the learning of corresponding skill policies to achieve those subgoals. Many approaches to subgoal discovery in HRL depend on the analysis of a model of the environment, but the need to learn such a model introduces its own problems of scale. Once subgoals are identified, skills may be learned through intrinsic motivation, introducing an internal reward signal marking subgoal attainment. In this paper, we present a novel model-free method for subgoal discovery using incremental unsupervised learning over a small memory of the
Authors
(none)
Tags
Stats
Related papers
- Efficient Exploration Through Intrinsic Motivation Learning For Unsupervised Subgoal Discovery In Model-free Hierarchical Reinforcement Learning (2019)0.00
- Exploring The Limits Of Hierarchical World Models In Reinforcement Learning (2024)6.34
- Generating Adjacency-constrained Subgoals In Hierarchical Reinforcement Learning (2020)0.00
- Goal Space Abstraction In Hierarchical Reinforcement Learning Via Reachability Analysis (2023)0.00
- Learning And Exploiting Multiple Subgoals For Fast Exploration In Hierarchical Reinforcement Learning (2019)0.00
- Multi-horizon Representations With Hierarchical Forward Models For Reinforcement Learning (2022)0.00
- Subgoal-based Hierarchical Reinforcement Learning For Multi-agent Collaboration (2024)0.00
- Near-optimal Representation Learning For Hierarchical Reinforcement Learning (2018)0.00