Hierarchical Decision Making Based On Structural Information Principles

Abstract

Hierarchical Reinforcement Learning (HRL) is a promising approach for managing task complexity across multiple levels of abstraction and accelerating long-horizon agent exploration. However, the effectiveness of hierarchical policies heavily depends on prior knowledge and manual assumptions about skill definitions and task decomposition. In this paper, we propose a novel Structural Information principles-based framework, namely SIDM, for hierarchical Decision Making in both single-agent and multi-agent scenarios. Central to our work is the utilization of structural information embedded in the decision-making process to adaptively and dynamically discover and learn hierarchical policies through environmental abstractions. Specifically, we present an abstraction mechanism that processes historical state-action trajectories to construct abstract representations of states and actions. We define and optimize directed structural entropy, a metric quantifying the uncertainty in transition dyn

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzeng2024hierarchical

Related papers

• Hierarchical Reinforcement Learning Via Advantage-weighted Information Maximization (2019)0.00
• Boosting Hierarchical Reinforcement Learning With Meta-learning For Complex Task Adaptation (2024)0.00
• MENTOR: Guiding Hierarchical Reinforcement Learning With Human Feedback And Dynamic Distance Constraint (2024)6.34
• Hierarchical Reinforcement Learning In Complex 3D Environments (2023)0.00
• Bidirectional-reachable Hierarchical Reinforcement Learning With Mutually Responsive Policies (2024)0.00
• Exploring The Limits Of Hierarchical World Models In Reinforcement Learning (2024)6.34
• Deep Reinforcement Learning From Hierarchical Preference Design (2023)2.00
• Learning Representations In Model-free Hierarchical Reinforcement Learning (2018)11.49