Goal Space Abstraction In Hierarchical Reinforcement Learning Via Reachability Analysis

Abstract

Open-ended learning benefits immensely from the use of symbolic methods for goal representation as they offer ways to structure knowledge for efficient and transferable learning. However, the existing Hierarchical Reinforcement Learning (HRL) approaches relying on symbolic reasoning are often limited as they require a manual goal representation. The challenge in autonomously discovering a symbolic goal representation is that it must preserve critical information, such as the environment dynamics. In this work, we propose a developmental mechanism for subgoal discovery via an emergent representation that abstracts (i.e., groups together) sets of environment states that have similar roles in the task. We create a HRL algorithm that gradually learns this representation along with the policies and evaluate it on navigation tasks to show the learned representation is interpretable and results in data efficiency.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzadem2023goal

Related papers

• Reconciling Spatial And Temporal Abstractions For Goal Representation (2024)1.69
• Learning Representations In Model-free Hierarchical Reinforcement Learning (2018)11.49
• Generating Adjacency-constrained Subgoals In Hierarchical Reinforcement Learning (2020)0.00
• Efficient Exploration Through Intrinsic Motivation Learning For Unsupervised Subgoal Discovery In Model-free Hierarchical Reinforcement Learning (2019)0.00
• Learning And Exploiting Multiple Subgoals For Fast Exploration In Hierarchical Reinforcement Learning (2019)0.00
• State-conditioned Adversarial Subgoal Generation (2022)0.00
• Exploring The Limits Of Hierarchical World Models In Reinforcement Learning (2024)6.34
• Near-optimal Representation Learning For Hierarchical Reinforcement Learning (2018)0.00