Temple: Learning Template Of Transitions For Sample Efficient Multi-task RL

Abstract

Transferring knowledge among various environments is important to efficiently learn multiple tasks online. Most existing methods directly use the previously learned models or previously learned optimal policies to learn new tasks. However, these methods may be inefficient when the underlying models or optimal policies are substantially different across tasks. In this paper, we propose Template Learning (TempLe), the first PAC-MDP method for multi-task reinforcement learning that could be applied to tasks with varying state/action space. TempLe generates transition dynamics templates, abstractions of the transition dynamics across tasks, to gain sample efficiency by extracting similarities between tasks even when their underlying models or optimal policies have limited commonalities. We present two algorithms for an "online" and a "finite-model" setting respectively. We prove that our proposed TempLe algorithms achieve much lower sample complexity than single-task learners or state-of-t

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keysun2020temple

Related papers

• A Model-based Approach For Sample-efficient Multi-task Reinforcement Learning (2019)0.00
• A Decentralized Policy Gradient Approach To Multi-task Reinforcement Learning (2020)0.00
• Harmodt: Harmony Multi-task Decision Transformer For Offline Reinforcement Learning (2024)0.00
• Provably Efficient Multi-task Reinforcement Learning With Model Transfer (2021)0.00
• Temporal Difference Models: Model-free Deep RL For Model-based Control (2018)0.00
• Double Meta-learning For Data Efficient Policy Optimization In Non-stationary Environments (2020)0.00
• Can RLHF Be More Efficient With Imperfect Reward Models? A Policy Coverage Perspective (2025)0.00
• TOM: Learning Policy-aware Models For Model-based Reinforcement Learning Via Transition Occupancy Matching (2023)0.00