Replay Buffer With Local Forgetting For Adapting To Local Environment Changes In Deep Model-based Reinforcement Learning

Abstract

One of the key behavioral characteristics used in neuroscience to determine whether the subject of study -- be it a rodent or a human -- exhibits model-based learning is effective adaptation to local changes in the environment, a particular form of adaptivity that is the focus of this work. In reinforcement learning, however, recent work has shown that modern deep model-based reinforcement-learning (MBRL) methods adapt poorly to local environment changes. An explanation for this mismatch is that MBRL methods are typically designed with sample-efficiency on a single task in mind and the requirements for effective adaptation are substantially higher, both in terms of the learned world model and the planning routine. One particularly challenging requirement is that the learned world model has to be sufficiently accurate throughout relevant parts of the state-space. This is challenging for deep-learning-based world models due to catastrophic forgetting. And while a replay buffer can mitiga

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrahimikalahroudi2023replay

Related papers

• Partial Models For Building Adaptive Model-based Reinforcement Learning Agents (2024)0.00
• Towards Evaluating Adaptivity Of Model-based Reinforcement Learning Methods (2022)0.00
• Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23
• The Loca Regret: A Consistent Metric To Evaluate Model-based Behavior In Reinforcement Learning (2020)0.00
• Replay-enhanced Continual Reinforcement Learning (2023)0.00
• Augmented Replay Memory In Reinforcement Learning With Continuous Control (2019)5.24
• Augmenting Replay In World Models For Continual Reinforcement Learning (2024)0.00
• Improving Generalization To New Environments And Removing Catastrophic Forgetting In Reinforcement Learning By Using An Eco-system Of Agents (2022)0.00