Episodic Curiosity Through Reachability

Abstract

Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning. In particular, inspired by curious behaviour in animals, observing something novel could be rewarded with a bonus. Such bonus is summed up with the real task reward - making it possible for RL algorithms to learn from the combined reward. We propose a new curiosity method which uses episodic memory to form the novelty bonus. To determine the bonus, the current observation is compared with the observations in memory. Crucially, the comparison is done based on how many environment steps it takes to reach the current observation from those in memory - which incorporates rich information about environment dynamics. This allows us to overcome the known "couch-potato" issues of prior work - when the agent finds a way to instantly gratif

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysavinov2018episodic

Related papers

• Curiosity-driven Multi-agent Exploration With Mixed Objectives (2022)0.00
• Is Curiosity All You Need? On The Utility Of Emergent Behaviours From Curious Exploration (2021)0.00
• Dynamic Memory-based Curiosity: A Bootstrap Approach For Exploration (2022)0.00
• Episodic Multi-agent Reinforcement Learning With Curiosity-driven Exploration (2021)0.00
• Wonder Wins Ways: Curiosity-driven Exploration Through Multi-agent Contextual Calibration (2025)0.00
• Rewarding Episodic Visitation Discrepancy For Exploration In Reinforcement Learning (2022)0.00
• Prototyping Three Key Properties Of Specific Curiosity In Computational Reinforcement Learning (2022)0.00
• Exploration Via Elliptical Episodic Bonuses (2022)3.58