Hypothesis-driven Skill Discovery For Hierarchical Deep Reinforcement Learning

Abstract

Deep reinforcement learning (DRL) is capable of learning high-performing policies on a variety of complex high-dimensional tasks, ranging from video games to robotic manipulation. However, standard DRL methods often suffer from poor sample efficiency, partially because they aim to be entirely problem-agnostic. In this work, we introduce a novel approach to exploration and hierarchical skill learning that derives its sample efficiency from intuitive assumptions it makes about the behavior of objects both in the physical world and simulations which mimic physics. Specifically, we propose the Hypothesis Proposal and Evaluation (HyPE) algorithm, which discovers objects from raw pixel data, generates hypotheses about the controllability of observed changes in object state, and learns a hierarchy of skills to test these hypotheses. We demonstrate that HyPE can dramatically improve the sample efficiency of policy learning in two different domains: a simulated robotic block-pushing domain, and

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keychuck2019hypothesis

Related papers

• Skill-critic: Refining Learned Skills For Hierarchical Reinforcement Learning (2023)7.50
• Disentangled Unsupervised Skill Discovery For Efficient Hierarchical Reinforcement Learning (2024)0.00
• Neuroevolution Is A Competitive Alternative To Reinforcement Learning For Skill Discovery (2022)0.00
• Learning Representations In Model-free Hierarchical Reinforcement Learning (2018)11.49
• Hierarchical Reinforcement Learning In Complex 3D Environments (2023)0.00
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction And Intrinsic Motivation (2016)0.00
• Hierarchical Reinforcement Learning With Advantage-based Auxiliary Rewards (2019)0.00
• Option Discovery In Hierarchical Reinforcement Learning Using Spatio-temporal Clustering (2016)0.00