Learn The Time To Learn: Replay Scheduling In Continual Learning

Abstract

Replay methods are known to be successful at mitigating catastrophic forgetting in continual learning scenarios despite having limited access to historical data. However, storing historical data is cheap in many real-world settings, yet replaying all historical data is often prohibited due to processing time constraints. In such settings, we propose that continual learning systems should learn the time to learn and schedule which tasks to replay at different time steps. We first demonstrate the benefits of our proposal by using Monte Carlo tree search to find a proper replay schedule, and show that the found replay schedules can outperform fixed scheduling policies when combined with various replay methods in different continual learning settings. Additionally, we propose a framework for learning replay scheduling policies with reinforcement learning. We show that the learned policies can generalize better in new continual learning scenarios compared to equally replaying all seen tasks

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyklasson2022learn

Related papers

• Replay-enhanced Continual Reinforcement Learning (2023)0.00
• Experience Replay For Continual Learning (2018)0.00
• Stable Continual Reinforcement Learning Via Diffusion-based Trajectory Replay (2024)0.00
• Replay For Safety (2021)0.00
• Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23
• Augmented Replay Memory In Reinforcement Learning With Continuous Control (2019)5.24
• Continual Learning Using World Models For Pseudo-rehearsal (2019)0.00
• Task-agnostic Continual Reinforcement Learning: Gaining Insights And Overcoming Challenges (2022)0.00