Scheduled Intrinsic Drive: A Hierarchical Take On Intrinsically Motivated Exploration

Abstract

Exploration in sparse reward reinforcement learning remains an open challenge. Many state-of-the-art methods use intrinsic motivation to complement the sparse extrinsic reward signal, giving the agent more opportunities to receive feedback during exploration. Commonly these signals are added as bonus rewards, which results in a mixture policy that neither conducts exploration nor task fulfillment resolutely. In this paper, we instead learn separate intrinsic and extrinsic task policies and schedule between these different drives to accelerate exploration and stabilize learning. Moreover, we introduce a new type of intrinsic reward denoted as successor feature control (SFC), which is general and not task-specific. It takes into account statistics over complete trajectories and thus differs from previous methods that only use local information to evaluate intrinsic motivation. We evaluate our proposed scheduled intrinsic drive (SID) agent using three different environments with pure visu

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2019scheduled

Related papers

• Coordinated Exploration Via Intrinsic Rewards For Multi-agent Reinforcement Learning (2019)0.00
• Successor-predecessor Intrinsic Exploration (2023)0.00
• Deep Intrinsically Motivated Exploration In Continuous Control (2022)0.00
• Self-supervised Exploration Via Temporal Inconsistency In Reinforcement Learning (2022)3.58
• Intrinsic Reward Policy Optimization For Sparse-reward Environments (2026)0.00
• A Unified Strategy For Implementing Curiosity And Empowerment Driven Reinforcement Learning (2018)0.00
• Information Content Exploration (2023)0.00
• Curiosity-driven Multi-agent Exploration With Mixed Objectives (2022)0.00