Goal Reaching With Eikonal-constrained Hierarchical Quasimetric Reinforcement Learning

Abstract

Goal-Conditioned Reinforcement Learning (GCRL) mitigates the difficulty of reward design by framing tasks as goal reaching rather than maximizing hand-crafted reward signals. In this setting, the optimal goal-conditioned value function naturally forms a quasimetric, motivating Quasimetric RL (QRL), which constrains value learning to quasimetric mappings and enforces local consistency through discrete, trajectory-based constraints. We propose Eikonal-Constrained Quasimetric RL (Eik-QRL), a continuous-time reformulation of QRL based on the Eikonal Partial Differential Equation (PDE). This PDE-based structure makes Eik-QRL trajectory-free, requiring only sampled states and goals, while improving out-of-distribution generalization. We provide theoretical guarantees for Eik-QRL and identify limitations that arise under complex dynamics. To address these challenges, we introduce Eik-Hierarchical QRL (Eik-HiQRL), which integrates Eik-QRL into a hierarchical decomposition. Empirically, Eik-HiQ

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygiammarino2025goal

Related papers

• Generating Adjacency-constrained Subgoals In Hierarchical Reinforcement Learning (2020)0.00
• Hindsight Expectation Maximization For Goal-conditioned Reinforcement Learning (2020)0.00
• Dual Goal Representations (2025)0.00
• Goal Space Abstraction In Hierarchical Reinforcement Learning Via Reachability Analysis (2023)0.00
• Contrastive Learning As Goal-conditioned Reinforcement Learning (2022)0.00
• State-conditioned Adversarial Subgoal Generation (2022)0.00
• Learning To Reach Goals Via Iterated Supervised Learning (2019)0.00
• Maximum Entropy-regularized Multi-goal Reinforcement Learning (2019)0.00