First Go, Then Post-explore: The Benefits Of Post-exploration In Intrinsic Motivation

Abstract

Go-Explore achieved breakthrough performance on challenging reinforcement learning (RL) tasks with sparse rewards. The key insight of Go-Explore was that successful exploration requires an agent to first return to an interesting state ('Go'), and only then explore into unknown terrain ('Explore'). We refer to such exploration after a goal is reached as 'post-exploration'. In this paper, we present a clear ablation study of post-exploration in a general intrinsically motivated goal exploration process (IMGEP) framework, that the Go-Explore paper did not show. We study the isolated potential of post-exploration, by turning it on and off within the same algorithm under both tabular and deep RL settings on both discrete navigation and continuous control tasks. Experiments on a range of MiniGrid and Mujoco environments show that post-exploration indeed helps IMGEP agents reach more diverse states and boosts their performance. In short, our work suggests that RL researchers should consider t

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2022first

Related papers

• Go-explore: A New Approach For Hard-exploration Problems (2019)0.00
• Explore-go: Leveraging Exploration For Generalisation In Deep Reinforcement Learning (2024)0.00
• Curiosity-driven Exploration In Sparse-reward Multi-agent Reinforcement Learning (2023)0.00
• Never Explore Repeatedly In Multi-agent Reinforcement Learning (2023)0.00
• The Impact Of Intrinsic Rewards On Exploration In Reinforcement Learning (2025)0.00
• Exploitation Is All You Need... For Exploration (2025)0.00
• Exploration And Incentives In Reinforcement Learning (2021)8.09
• Redeeming Intrinsic Rewards Via Constrained Optimization (2022)0.00