Accelerating Multi-task Temporal Difference Learning Under Low-rank Representation

Abstract

We study policy evaluation problems in multi-task reinforcement learning (RL) under a low-rank representation setting. In this setting, we are given \(N\) learning tasks where the corresponding value function of these tasks lie in an \(r\)-dimensional subspace, with \(r<N\). One can apply the classic temporal-difference (TD) learning method for solving these problems where this method learns the value function of each task independently. In this paper, we are interested in understanding whether one can exploit the low-rank structure of the multi-task setting to accelerate the performance of TD learning. To answer this question, we propose a new variant of TD learning method, where we integrate the so-called truncated singular value decomposition step into the update of TD learning. This additional step will enable TD learning to exploit the dominant directions due to the low rank structure to update the iterates, therefore, improving its performance. Our empirical results show that the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybai2025accelerating

Related papers

• Provable Multi-task Reinforcement Learning: A Representation Learning Framework With Low Rank Rewards (2026)0.00
• Provable Benefit Of Multitask Representation Learning In Reinforcement Learning (2022)0.00
• Discerning Temporal Difference Learning (2023)0.00
• Learning Sparse Representations In Reinforcement Learning (2019)0.00
• Multilinear Tensor Low-rank Approximation For Policy-gradient Methods In Reinforcement Learning (2025)0.00
• Finite-sample Analysis Of Decentralized Temporal-difference Learning With Linear Function Approximation (2019)0.00
• Adaptive Temporal-difference Learning For Policy Evaluation With Per-state Uncertainty Estimates (2019)0.00
• Matrix Estimation For Offline Reinforcement Learning With Low-rank Structure (2023)0.00