Reconciling Spatial And Temporal Abstractions For Goal Representation

Abstract

Goal representation affects the performance of Hierarchical Reinforcement Learning (HRL) algorithms by decomposing the complex learning problem into easier subtasks. Recent studies show that representations that preserve temporally abstract environment dynamics are successful in solving difficult problems and provide theoretical guarantees for optimality. These methods however cannot scale to tasks where environment dynamics increase in complexity i.e. the temporally abstract transition relations depend on larger number of variables. On the other hand, other efforts have tried to use spatial abstraction to mitigate the previous issues. Their limitations include scalability to high dimensional environments and dependency on prior knowledge. In this paper, we propose a novel three-layer HRL algorithm that introduces, at different levels of the hierarchy, both a spatial and a temporal goal abstraction. We provide a theoretical study of the regret bounds of the learned policies. We evalu

Stats

• citations0
• S2 citations—
• github stars6
• HF likes0
• heat score1.69
• arxiv keyzadem2024reconciling

Related papers

• Goal Space Abstraction In Hierarchical Reinforcement Learning Via Reachability Analysis (2023)0.00
• Learning Representations In Model-free Hierarchical Reinforcement Learning (2018)11.49
• Generating Adjacency-constrained Subgoals In Hierarchical Reinforcement Learning (2020)0.00
• Exploring The Limits Of Hierarchical World Models In Reinforcement Learning (2024)6.34
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction And Intrinsic Motivation (2016)0.00
• Subgoal-based Hierarchical Reinforcement Learning For Multi-agent Collaboration (2024)0.00
• State-conditioned Adversarial Subgoal Generation (2022)0.00
• Learning And Exploiting Multiple Subgoals For Fast Exploration In Hierarchical Reinforcement Learning (2019)0.00