Using Curiosity For An Even Representation Of Tasks In Continual Offline Reinforcement Learning

Abstract

In this work, we investigate the means of using curiosity on replay buffers to improve offline multi-task continual reinforcement learning when tasks, which are defined by the non-stationarity in the environment, are non labeled and not evenly exposed to the learner in time. In particular, we investigate the use of curiosity both as a tool for task boundary detection and as a priority metric when it comes to retaining old transition tuples, which we respectively use to propose two different buffers. Firstly, we propose a Hybrid Reservoir Buffer with Task Separation (HRBTS), where curiosity is used to detect task boundaries that are not known due to the task agnostic nature of the problem. Secondly, by using curiosity as a priority metric when it comes to retaining old transition tuples, a Hybrid Curious Buffer (HCB) is proposed. We ultimately show that these buffers, in conjunction with regular reinforcement learning algorithms, can be used to alleviate the catastrophic forgetting issu

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypathmanathan2023using

Related papers

• OER: Offline Experience Replay For Continual Offline Reinforcement Learning (2023)3.58
• CUDC: A Curiosity-driven Unsupervised Data Collection Method With Adaptive Temporal Distances For Offline Reinforcement Learning (2023)2.26
• Replay-enhanced Continual Reinforcement Learning (2023)0.00
• Episodic Curiosity Through Reachability (2018)0.00
• Continual Offline Reinforcement Learning Via Diffusion-based Dual Generative Replay (2024)0.00
• Adaptive Replay Buffer For Offline-to-online Reinforcement Learning (2025)0.00
• Tsn-affinity: Similarity-driven Parameter Reuse For Continual Offline Reinforcement Learning (2026)0.00
• Dynamic Memory-based Curiosity: A Bootstrap Approach For Exploration (2022)0.00