On The Performance Of Temporal Difference Learning With Neural Networks

Abstract

Neural Temporal Difference (TD) Learning is an approximate temporal difference method for policy evaluation that uses a neural network for function approximation. Analysis of Neural TD Learning has proven to be challenging. In this paper we provide a convergence analysis of Neural TD Learning with a projection onto \(B(\theta_0, \omega)\), a ball of fixed radius \(\omega\) around the initial point \(\theta_0\). We show an approximation bound of \(O(\epsilon) + \tilde\{O\} (1/\sqrt\{m\})\) where \(\epsilon\) is the approximation quality of the best neural network in \(B(\theta_0, \omega)\) and \(m\) is the width of all hidden layers in the network.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytian2023on

Related papers

• An Improved Finite-time Analysis Of Temporal Difference Learning With Deep Neural Networks (2024)0.00
• Adaptive Temporal Difference Learning With Linear Function Approximation (2020)0.00
• Sample Complexity And Overparameterization Bounds For Temporal Difference Learning With Neural Network Approximation (2021)0.00
• A Finite Time Analysis Of Temporal Difference Learning With Linear Function Approximation (2018)0.00
• Approximate Temporal Difference Learning Is A Gradient Descent For Reversible Policies (2018)0.00
• Neural Temporal-difference And Q-learning Provably Converge To Global Optima (2019)7.81
• Preferential Temporal Difference Learning (2021)0.00
• Finite-time Performance Of Distributed Temporal Difference Learning With Linear Function Approximation (2019)9.59