Same State, Different Task: Continual Reinforcement Learning Without Interference

Abstract

Continual Learning (CL) considers the problem of training an agent sequentially on a set of tasks while seeking to retain performance on all previous tasks. A key challenge in CL is catastrophic forgetting, which arises when performance on a previously mastered task is reduced when learning a new task. While a variety of methods exist to combat forgetting, in some cases tasks are fundamentally incompatible with each other and thus cannot be learnt by a single policy. This can occur, in reinforcement learning (RL) when an agent may be rewarded for achieving different goals from the same observation. In this paper we formalize this "interference" as distinct from the problem of forgetting. We show that existing CL methods based on single neural network predictors with shared replay buffers fail in the presence of interference. Instead, we propose a simple method, OWL, to address this challenge. OWL learns a factorized policy, using shared feature extraction layers, but separate heads, ea

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykessler2021same

Related papers

• Task-agnostic Continual Reinforcement Learning: Gaining Insights And Overcoming Challenges (2022)0.00
• Continual Policy Distillation From Distributed Reinforcement Learning Teachers (2026)0.00
• Replay-enhanced Continual Reinforcement Learning (2023)0.00
• Continual World: A Robotic Benchmark For Continual Reinforcement Learning (2021)0.00
• Multi-task Learning And Catastrophic Forgetting In Continual Reinforcement Learning (2019)0.00
• Continual Reinforcement Learning By Planning With Online World Models (2025)0.00
• Online Reinforcement Learning In Non-stationary Context-driven Environments (2023)0.00
• Experience Replay For Continual Learning (2018)0.00