Reset & Distill: A Recipe For Overcoming Negative Transfer In Continual Reinforcement Learning

Abstract

We argue that the negative transfer problem occurring when the new task to learn arrives is an important problem that needs not be overlooked when developing effective Continual Reinforcement Learning (CRL) algorithms. Through comprehensive experimental validation, we demonstrate that such issue frequently exists in CRL and cannot be effectively addressed by several recent work on either mitigating plasticity loss of RL agents or enhancing the positive transfer in CRL scenario. To that end, we develop Reset & Distill (R&D), a simple yet highly effective baseline method, to overcome the negative transfer problem in CRL. R&D combines a strategy of resetting the agent's online actor and critic networks to learn a new task and an offline learning step for distilling the knowledge from the online actor and previous expert's action probabilities. We carried out extensive experiments on long sequence of Meta World tasks and show that our simple baseline method consistently outperforms recent

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyahn2024reset

Related papers

• Continual Policy Distillation From Distributed Reinforcement Learning Teachers (2026)0.00
• Task-agnostic Continual Reinforcement Learning: Gaining Insights And Overcoming Challenges (2022)0.00
• Continual World: A Robotic Benchmark For Continual Reinforcement Learning (2021)0.00
• Replay-enhanced Continual Reinforcement Learning (2023)0.00
• Continual Deep Reinforcement Learning With Task-agnostic Policy Distillation (2024)0.00
• Multi-granularity Knowledge Transfer For Continual Reinforcement Learning (2024)2.26
• Fine-tuning Reinforcement Learning Models Is Secretly A Forgetting Mitigation Problem (2024)0.00
• Same State, Different Task: Continual Reinforcement Learning Without Interference (2021)0.00