Modular Continual Learning In A Unified Visual Environment

Abstract

A core aspect of human intelligence is the ability to learn new tasks quickly and switch between them flexibly. Here, we describe a modular continual reinforcement learning paradigm inspired by these abilities. We first introduce a visual interaction environment that allows many types of tasks to be unified in a single framework. We then describe a reward map prediction scheme that learns new tasks robustly in the very large state and action spaces required by such an environment. We investigate how properties of module architecture influence efficiency of task learning, showing that a module motif incorporating specific design principles (e.g. early bottlenecks, low-order polynomial nonlinearities, and symmetry) significantly outperforms more standard neural network motifs, needing fewer training examples and fewer neurons to achieve high levels of performance. Finally, we present a meta-controller architecture for task switching based on a dynamic neural voting scheme, which allows n

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfeigelis2017modular

Related papers

• Lifelong Reinforcement Learning Via Neuromodulation (2024)0.00
• Modularization Of End-to-end Learning: Case Study In Arcade Games (2019)0.00
• Modularity Benefits Reinforcement Learning Agents With Competing Homeostatic Drives (2022)0.00
• Continual Visual Reinforcement Learning With A Life-long World Model (2023)2.26
• Self-activating Neural Ensembles For Continual Reinforcement Learning (2022)0.00
• Task-agnostic Continual Reinforcement Learning: Gaining Insights And Overcoming Challenges (2022)0.00
• Improving Reinforcement Learning Efficiency With Auxiliary Tasks In Non-visual Environments: A Comparison (2023)2.26
• Modular Networks Prevent Catastrophic Interference In Model-based Multi-task Reinforcement Learning (2021)0.00