Accelerated Distributional Temporal Difference Learning With Linear Function Approximation

Abstract

In this paper, we study the finite-sample statistical rates of distributional temporal difference (TD) learning with linear function approximation. The purpose of distributional TD learning is to estimate the return distribution of a discounted Markov decision process for a given policy. Previous works on statistical analysis of distributional TD learning focus mainly on the tabular case. We first consider the linear function approximation setting and conduct a fine-grained analysis of the linear-categorical Bellman equation. Building on this analysis, we further incorporate variance reduction techniques in our new algorithms to establish tight sample complexity bounds independent of the support size \(K\) when \(K\) is large. Our theoretical results imply that, when employing distributional TD learning with linear function approximation, learning the full distribution of the return function from streaming data is no more difficult than learning its expectation. This work provide new i

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjin2025accelerated

Related papers

• A Finite Sample Analysis Of Distributional TD Learning With Linear Function Approximation (2025)0.00
• A Finite Time Analysis Of Temporal Difference Learning With Linear Function Approximation (2018)0.00
• Finite-time Performance Of Distributed Temporal Difference Learning With Linear Function Approximation (2019)9.59
• Adaptive Temporal Difference Learning With Linear Function Approximation (2020)0.00
• Stability And Sensitivity Analysis Of Relative Temporal-difference Learning: Extended Version (2026)0.00
• Finite Sample Analysis Of Linear Temporal Difference Learning With Arbitrary Features (2025)0.00
• Finite-sample Analysis Of Decentralized Temporal-difference Learning With Linear Function Approximation (2019)0.00
• Nonlinear Distributional Gradient Temporal-difference Learning (2018)0.00