Improvements Of Dark Experience Replay And Reservoir Sampling Towards Better Balance Between Consolidation And Plasticity

Abstract

Continual learning is the one of the most essential abilities for autonomous agents, which can incrementally learn daily-life skills. For this ultimate goal, a simple but powerful method, dark experience replay (DER), has been proposed recently. DER mitigates catastrophic forgetting, in which the skills acquired in the past are unintentionally forgotten, by stochastically storing the streaming data in a reservoir sampling (RS) buffer and by relearning them or retaining the past outputs for them. However, since DER considers multiple objectives, it will not function properly without appropriate weighting of them. In addition, the ability to retain past outputs inhibits learning if the past outputs are incorrect due to distribution shift or other effects. This is due to a tradeoff between memory consolidation and plasticity. The tradeoff is hidden even in the RS buffer, which gradually stops storing new data for new skills in it as data is continuously passed to it. To alleviate the trad

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykobayashi2025improvements

Related papers

• Replay-enhanced Continual Reinforcement Learning (2023)0.00
• Augmented Replay Memory In Reinforcement Learning With Continuous Control (2019)5.24
• Experience Replay For Continual Learning (2018)0.00
• Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23
• Stabilising Experience Replay For Deep Multi-agent Reinforcement Learning (2017)0.00
• Bootstrapping A DQN Replay Memory With Synthetic Experiences (2020)5.84
• Lucid Dreaming For Experience Replay: Refreshing Past States With The Current Policy (2020)7.81
• Prioritized Generative Replay (2024)0.00