Continual Visual Reinforcement Learning With A Life-long World Model

Abstract

Learning physical dynamics in a series of non-stationary environments is a challenging but essential task for model-based reinforcement learning (MBRL) with visual inputs. It requires the agent to consistently adapt to novel tasks without forgetting previous knowledge. In this paper, we present a new continual learning approach for visual dynamics modeling and explore its efficacy in visual control. The key assumption is that an ideal world model can provide a non-forgetting environment simulator, which enables the agent to optimize the policy in a multi-task learning manner based on the imagined trajectories from the world model. To this end, we first introduce the life-long world model, which learns task-specific latent dynamics using a mixture of Gaussians and incorporates generative experience replay to mitigate catastrophic forgetting. Then, we further address the value estimation challenge for previous tasks with the exploratory-conservative behavior learning approach. Our model

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keypan2023continual

Related papers

• Continual Learning Using World Models For Pseudo-rehearsal (2019)0.00
• Continual Reinforcement Learning By Planning With Online World Models (2025)0.00
• The Effectiveness Of World Models For Continual Reinforcement Learning (2022)0.00
• Model-based Reinforcement Learning For Control Under Time-varying Dynamics (2026)0.00
• World Models As Reference Trajectories For Rapid Motor Adaptation (2025)0.00
• Enhancing Vision-language Model Training With Reinforcement Learning In Synthetic Worlds For Real-world Success (2025)0.00
• Continual Reinforcement Learning In 3D Non-stationary Environments (2019)0.00
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89