Classifying Options For Deep Reinforcement Learning

Abstract

In this paper we combine one method for hierarchical reinforcement learning - the options framework - with deep Q-networks (DQNs) through the use of different "option heads" on the policy network, and a supervisory network for choosing between the different options. We utilise our setup to investigate the effects of architectural constraints in subtasks with positive and negative transfer, across a range of network capacities. We empirically show that our augmented DQN has lower sample complexity when simultaneously learning subtasks with negative transfer, without degrading performance when learning subtasks with positive transfer.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyarulkumaran2016classifying

Related papers

• Interpretable Option Discovery Using Deep Q-learning And Variational Autoencoders (2022)0.00
• Hyperbolic Embeddings For Learning Options In Hierarchical Reinforcement Learning (2018)0.00
• Modular Multi-objective Deep Reinforcement Learning With Decision Values (2017)10.74
• Discovering Hierarchies Using Imitation Learning From Hierarchy Aware Policies (2018)0.00
• Reusable Options Through Gradient-based Meta Learning (2022)0.00
• Deep Multi-agent Reinforcement Learning With Discrete-continuous Hybrid Action Spaces (2019)12.47
• A Provably Efficient Option-based Algorithm For Both High-level And Low-level Learning (2024)0.00
• Autonomous Option Invention For Continual Hierarchical Reinforcement Learning And Planning (2024)2.26