Discovering Hierarchies Using Imitation Learning From Hierarchy Aware Policies

Abstract

Learning options that allow agents to exhibit temporally higher order behavior has proven to be useful in increasing exploration, reducing sample complexity and for various transfer scenarios. Deep Discovery of Options (DDO) is a generative algorithm that learns a hierarchical policy along with options directly from expert trajectories. We perform a qualitative and quantitative analysis of options inferred from DDO in different domains. To this end, we suggest different value metrics like option termination condition, hinge value function error and KL-Divergence based distance metric to compare different methods. Analyzing the termination condition of the options and number of time steps the options were run revealed that the options were terminating prematurely. We suggest modifications which can be incorporated easily and alleviates the problem of shorter options and a collapse of options to the same mode.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydeshpande2018discovering

Related papers

• Learning And Exploiting Multiple Subgoals For Fast Exploration In Hierarchical Reinforcement Learning (2019)0.00
• Provable Hierarchical Imitation Learning Via EM (2020)0.00
• Reusable Options Through Gradient-based Meta Learning (2022)0.00
• A Provably Efficient Option-based Algorithm For Both High-level And Low-level Learning (2024)0.00
• Offline Hierarchical Reinforcement Learning Via Inverse Optimization (2024)0.00
• Classifying Options For Deep Reinforcement Learning (2016)0.00
• Reinforcement Learning In Pomdps With Memoryless Options And Option-observation Initiation Sets (2017)6.77
• Hierarchical Reinforcement Learning Via Advantage-weighted Information Maximization (2019)0.00