Concentration Of Contractive Stochastic Approximation And Reinforcement Learning

Abstract

Using a martingale concentration inequality, concentration bounds `from time \(n_0\) on' are derived for stochastic approximation algorithms with contractive maps and both martingale difference and Markov noises. These are applied to reinforcement learning algorithms, in particular to asynchronous Q-learning and TD(0).

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychandak2021concentration

Related papers

• Almost Sure Convergence Rates And Concentration Of Stochastic Approximation And Reinforcement Learning With Markovian Noise (2024)0.00
• Uncertainty Quantification For Markov Chain Induced Martingales With Application To Temporal Difference Learning (2025)0.00
• The ODE Method For Stochastic Approximation And Reinforcement Learning With Markovian Noise (2024)0.00
• Finite-sample Analysis Of Nonlinear Stochastic Approximation With Applications In Reinforcement Learning (2019)10.35
• Central Limit Theorem For Two-timescale Stochastic Approximation With Markovian Noise: Theory And Applications (2024)0.00
• Finite Time Analysis Of Linear Two-timescale Stochastic Approximation With Markovian Noise (2020)0.00
• On The Convergence Of Consensus Algorithms With Markovian Noise And Gradient Bias (2020)0.00
• Conjugated Discrete Distributions For Distributional Reinforcement Learning (2021)0.00