Disentangled Unsupervised Skill Discovery For Efficient Hierarchical Reinforcement Learning

Abstract

A hallmark of intelligent agents is the ability to learn reusable skills purely from unsupervised interaction with the environment. However, existing unsupervised skill discovery methods often learn entangled skills where one skill variable simultaneously influences many entities in the environment, making downstream skill chaining extremely challenging. We propose Disentangled Unsupervised Skill Discovery (DUSDi), a method for learning disentangled skills that can be efficiently reused to solve downstream tasks. DUSDi decomposes skills into disentangled components, where each skill component only affects one factor of the state space. Importantly, these skill components can be concurrently composed to generate low-level actions, and efficiently chained to tackle downstream tasks through hierarchical Reinforcement Learning. DUSDi defines a novel mutual-information-based objective to enforce disentanglement between the influences of different skill components, and utilizes value factori

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhu2024disentangled

Related papers

• Skild: Unsupervised Skill Discovery Guided By Factor Interactions (2024)0.00
• Hypothesis-driven Skill Discovery For Hierarchical Deep Reinforcement Learning (2019)2.26
• Option Discovery In Hierarchical Reinforcement Learning Using Spatio-temporal Clustering (2016)0.00
• Hierarchical Cooperative Multi-agent Reinforcement Learning With Skill Discovery (2019)5.24
• Diversity Is All You Need: Learning Skills Without A Reward Function (2018)0.00
• Focused Skill Discovery: Learning To Control Specific State Variables While Minimizing Side Effects (2025)0.00
• Distop: Discovering A Topological Representation To Learn Diverse And Rewarding Skills (2021)3.58
• Skills: Adaptive Skill Sequencing For Efficient Temporally-extended Exploration (2022)0.00