Investigating The Role Of Model-based Learning In Exploration And Transfer

Abstract

State of the art reinforcement learning has enabled training agents on tasks of ever increasing complexity. However, the current paradigm tends to favor training agents from scratch on every new task or on collections of tasks with a view towards generalizing to novel task configurations. The former suffers from poor data efficiency while the latter is difficult when test tasks are out-of-distribution. Agents that can effectively transfer their knowledge about the world pose a potential solution to these issues. In this paper, we investigate transfer learning in the context of model-based agents. Specifically, we aim to understand when exactly environment models have an advantage and why. We find that a model-based approach outperforms controlled model-free baselines for transfer learning. Through ablations, we show that both the policy and dynamics model learnt through exploration matter for successful transfer. We demonstrate our results across three domains which vary in their requi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywalker2023investigating

Related papers

• The Role Of Exploration For Task Transfer In Reinforcement Learning (2022)0.00
• Is Exploration All You Need? Effective Exploration Characteristics For Transfer In Reinforcement Learning (2024)0.00
• Extra: Transfer-guided Exploration (2019)0.00
• Off-policy Reinforcement Learning With Model-based Exploration Augmentation (2025)0.00
• Contextual Policy Transfer In Reinforcement Learning Domains Via Deep Mixtures-of-experts (2020)0.00
• Interesting Object, Curious Agent: Learning Task-agnostic Exploration (2021)0.00
• Explore-go: Leveraging Exploration For Generalisation In Deep Reinforcement Learning (2024)0.00
• Exploration In Knowledge Transfer Utilizing Reinforcement Learning (2024)0.00