A Model-based Approach For Sample-efficient Multi-task Reinforcement Learning

Abstract

The aim of multi-task reinforcement learning is two-fold: (1) efficiently learn by training against multiple tasks and (2) quickly adapt, using limited samples, to a variety of new tasks. In this work, the tasks correspond to reward functions for environments with the same (or similar) dynamical models. We propose to learn a dynamical model during the training process and use this model to perform sample-efficient adaptation to new tasks at test time. We use significantly fewer samples by performing policy optimization only in a "virtual" environment whose transitions are given by our learned dynamical model. Our algorithm sequentially trains against several tasks. Upon encountering a new task, we first warm-up a policy on our learned dynamical model, which requires no new samples from the environment. We then adapt the dynamical model with samples from this policy in the real environment. We evaluate our approach on several continuous control benchmarks and demonstrate its efficacy ov

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylandolfi2019a

Related papers

• Double Meta-learning For Data Efficient Policy Optimization In Non-stationary Environments (2020)0.00
• Towards A Simple Approach To Multi-step Model-based Reinforcement Learning (2018)0.00
• Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning With Diverse Tasks (2024)0.00
• Harmonydream: Task Harmonization Inside World Models (2023)3.46
• Model-based Adaptation For Sample Efficient Transfer In Reinforcement Learning Control Of Parameter-varying Systems (2023)2.26
• Modular Networks Prevent Catastrophic Interference In Model-based Multi-task Reinforcement Learning (2021)0.00
• PWM: Policy Learning With Multi-task World Models (2024)0.00
• A Decentralized Policy Gradient Approach To Multi-task Reinforcement Learning (2020)0.00