Continual Learning Using World Models For Pseudo-rehearsal

Abstract

The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experience to be segmented and labeled as discrete tasks, however, in continuous experience it is generally unclear what a sufficient segmentation of tasks would be. Here we propose a method to continually learn these internal world models through the interleaving of internally generated episodes of past experiences (i.e., pseudo-rehearsal). We show this method can sequentially learn unsupervised temporal prediction, without task labels, in a disparate set of Atari games. Empirically, this interleaving of the internally generated rollouts with the external environment's observations leads to a consistent reduction in temporal prediction loss compared to non-interleaved learning

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyketz2019continual

Related papers

• The Effectiveness Of World Models For Continual Reinforcement Learning (2022)0.00
• Continual Visual Reinforcement Learning With A Life-long World Model (2023)2.26
• Augmenting Replay In World Models For Continual Reinforcement Learning (2024)0.00
• Experience Replay For Continual Learning (2018)0.00
• Continual State Representation Learning For Reinforcement Learning Using Generative Replay (2018)0.00
• Replay-enhanced Continual Reinforcement Learning (2023)0.00
• Continual Reinforcement Learning By Planning With Online World Models (2025)0.00
• Learning To Predict Without Looking Ahead: World Models Without Forward Prediction (2019)0.00