Provable Benefits Of Multi-task RL Under Non-markovian Decision Making Processes

Abstract

In multi-task reinforcement learning (RL) under Markov decision processes (MDPs), the presence of shared latent structures among multiple MDPs has been shown to yield significant benefits to the sample efficiency compared to single-task RL. In this paper, we investigate whether such a benefit can extend to more general sequential decision making problems, such as partially observable MDPs (POMDPs) and more general predictive state representations (PSRs). The main challenge here is that the large and complex model space makes it hard to identify what types of common latent structure of multi-task PSRs can reduce the model complexity and improve sample efficiency. To this end, we posit a joint model class for tasks and use the notion of \(\eta\)-bracketing number to quantify its complexity; this number also serves as a general metric to capture the similarity of tasks and thus determines the benefit of multi-task over single-task RL. We first study upstream multi-task learning over PSRs,

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhuang2023provable

Related papers

• Provable Benefit Of Multitask Representation Learning In Reinforcement Learning (2022)0.00
• Towards Understanding The Benefit Of Multitask Representation Learning In Decision Process (2025)0.00
• Provably Efficient Multi-task Reinforcement Learning With Model Transfer (2021)0.00
• Provable Multi-task Reinforcement Learning: A Representation Learning Framework With Low Rank Rewards (2026)0.00
• Learning Robust State Abstractions For Hidden-parameter Block Mdps (2020)0.00
• Group Distributionally Robust Reinforcement Learning With Hierarchical Latent Variables (2022)0.00
• Optimal Decision-making In Mixed-agent Partially Observable Stochastic Environments Via Reinforcement Learning (2019)0.00
• Provably Efficient Ucb-type Algorithms For Learning Predictive State Representations (2023)0.00