Experience Replay For Continual Learning

Abstract

Continual learning is the problem of learning new tasks or knowledge while protecting old knowledge and ideally generalizing from old experience to learn new tasks faster. Neural networks trained by stochastic gradient descent often degrade on old tasks when trained successively on new tasks with different data distributions. This phenomenon, referred to as catastrophic forgetting, is considered a major hurdle to learning with non-stationary data or sequences of new tasks, and prevents networks from continually accumulating knowledge and skills. We examine this issue in the context of reinforcement learning, in a setting where an agent is exposed to tasks in a sequence. Unlike most other work, we do not provide an explicit indication to the model of task boundaries, which is the most general circumstance for a learning agent exposed to continuous experience. While various methods to counteract catastrophic forgetting have recently been proposed, we explore a straightforward, general, a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrolnick2018experience

Related papers

• Replay-enhanced Continual Reinforcement Learning (2023)0.00
• Learn The Time To Learn: Replay Scheduling In Continual Learning (2022)0.00
• Stable Continual Reinforcement Learning Via Diffusion-based Trajectory Replay (2024)0.00
• Augmented Replay Memory In Reinforcement Learning With Continuous Control (2019)5.24
• Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23
• Continual Learning Using World Models For Pseudo-rehearsal (2019)0.00
• Continual Offline Reinforcement Learning Via Diffusion-based Dual Generative Replay (2024)0.00
• Continual Reinforcement Learning With Complex Synapses (2018)0.00