Overcoming Catastrophic Interference In Online Reinforcement Learning With Dynamic Self-organizing Maps
2019 · Yat Long Lo, Sina Ghiassian
Abstract
Using neural networks in the reinforcement learning (RL) framework has achieved notable successes. Yet, neural networks tend to forget what they learned in the past, especially when they learn online and fully incrementally, a setting in which the weights are updated after each sample is received and the sample is then discarded. Under this setting, an update can lead to overly global generalization by changing too many weights. The global generalization interferes with what was previously learned and deteriorates performance, a phenomenon known as catastrophic interference. Many previous works use mechanisms such as experience replay (ER) buffers to mitigate interference by performing minibatch updates, ensuring the data distribution is approximately independent-and-identically-distributed (i.i.d.). But using ER would become infeasible in terms of memory as problem complexity increases. Thus, it is crucial to look for more memory-efficient alternatives. Interference can be averted if
Authors
(none)
Tags
Stats
Related papers
- Memory-efficient Episodic Control Reinforcement Learning With Dynamic Online K-means (2019)0.00
- Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23
- Meta-reinforcement Learning With Self-modifying Networks (2022)4.52
- Improving Generalization To New Environments And Removing Catastrophic Forgetting In Reinforcement Learning By Using An Eco-system Of Agents (2022)0.00
- An Intentional Forgetting-driven Self-healing Method For Deep Reinforcement Learning Systems (2023)0.00
- Stabilising Experience Replay For Deep Multi-agent Reinforcement Learning (2017)0.00
- Replay Buffer With Local Forgetting For Adapting To Local Environment Changes In Deep Model-based Reinforcement Learning (2023)0.00
- Learning To Cope With Adversarial Attacks (2019)0.00