Deep Online Learning Via Meta-learning: Continual Adaptation For Model-based RL

Abstract

Humans and animals can learn complex predictive models that allow them to accurately and reliably reason about real-world phenomena, and they can adapt such models extremely quickly in the face of unexpected changes. Deep neural network models allow us to represent very complex functions, but lack this capacity for rapid online adaptation. The goal in this paper is to develop a method for continual online learning from an incoming stream of data, using deep neural network models. We formulate an online learning procedure that uses stochastic gradient descent to update model parameters, and an expectation maximization algorithm with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distributions. This allows for all models to be adapted as necessary, with new models instantiated for task changes and old models recalled when previously seen tasks are encountered again. Furthermore, we observe that meta-learning can be used to met

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynagabandi2018deep

Related papers

• Learning To Reinforcement Learn (2016)0.00
• Meta-gradient Reinforcement Learning With An Objective Discovered Online (2020)0.00
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Lifelong Reinforcement Learning Via Neuromodulation (2024)0.00
• Principled Fast And Meta Knowledge Learners For Continual Reinforcement Learning (2026)0.00
• Context Meta-reinforcement Learning Via Neuromodulation (2021)6.34
• Boosting Hierarchical Reinforcement Learning With Meta-learning For Complex Task Adaptation (2024)0.00
• Offline Meta-reinforcement Learning With Online Self-supervision (2021)0.00