On The Feasibility Of Cross-task Transfer With Model-based Reinforcement Learning

Abstract

Reinforcement Learning (RL) algorithms can solve challenging control problems directly from image observations, but they often require millions of environment interactions to do so. Recently, model-based RL algorithms have greatly improved sample-efficiency by concurrently learning an internal model of the world, and supplementing real environment interactions with imagined rollouts for policy improvement. However, learning an effective model of the world from scratch is challenging, and in stark contrast to humans that rely heavily on world understanding and visual cues for learning new skills. In this work, we investigate whether internal models learned by modern model-based RL algorithms can be leveraged to solve new, distinctly different tasks faster. We propose Model-Based Cross-Task Transfer (XTRA), a framework for sample-efficient online RL with scalable pretraining and finetuning of learned world models. By offline multi-task pretraining and online cross-task finetuning, we ach

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxu2022on

Related papers

• Self-supervised Reinforcement Learning That Transfers Using Random Features (2023)2.26
• Zero-shot Cross-modal Transfer Of Reinforcement Learning Policies Through A Global Workspace (2024)0.00
• Human-inspired Framework To Accelerate Reinforcement Learning (2023)0.00
• A Model-based Approach For Sample-efficient Multi-task Reinforcement Learning (2019)0.00
• Improving Reinforcement Learning Efficiency With Auxiliary Tasks In Non-visual Environments: A Comparison (2023)2.26
• Investigating The Role Of Model-based Learning In Exploration And Transfer (2023)0.00
• Provably Efficient Multi-task Reinforcement Learning With Model Transfer (2021)0.00
• On Mechanisms For Transfer Using Landmark Value Functions In Multi-task Lifelong Reinforcement Learning (2019)0.00