Self-organizing Maps As A Storage And Transfer Mechanism In Reinforcement Learning

Abstract

The idea of reusing information from previously learned tasks (source tasks) for the learning of new tasks (target tasks) has the potential to significantly improve the sample efficiency reinforcement learning agents. In this work, we describe an approach to concisely store and represent learned task knowledge, and reuse it by allowing it to guide the exploration of an agent while it learns new tasks. In order to do so, we use a measure of similarity that is defined directly in the space of parameterized representations of the value functions. This similarity measure is also used as a basis for a variant of the growing self-organizing map algorithm, which is simultaneously used to enable the storage of previously acquired task knowledge in an adaptive and scalable manner.We empirically validate our approach in a simulated navigation environment and discuss possible extensions to this approach along with potential applications where it could be particularly useful.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykarimpanal2018self

Related papers

• Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23
• Overcoming Catastrophic Interference In Online Reinforcement Learning With Dynamic Self-organizing Maps (2019)0.00
• Distill Knowledge In Multi-task Reinforcement Learning With Optimal-transport Regularization (2023)0.00
• Intrinsic Rewards From Self-organizing Feature Maps For Exploration In Reinforcement Learning (2023)0.00
• Skills: Adaptive Skill Sequencing For Efficient Temporally-extended Exploration (2022)0.00
• Similarity-based Knowledge Transfer For Cross-domain Reinforcement Learning (2023)0.00
• Self-adapting Goals Allow Transfer Of Predictive Models To New Tasks (2019)2.26
• Intrinsically Motivated Self-supervised Learning In Reinforcement Learning (2021)3.58