Q-learning As A Monotone Scheme

Abstract

Stability issues with reinforcement learning methods persist. To better understand some of these stability and convergence issues involving deep reinforcement learning methods, we examine a simple linear quadratic example. We interpret the convergence criterion of exact Q-learning in the sense of a monotone scheme and discuss consequences of function approximation on monotonicity properties.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2024q

Related papers

• Stabilizing Q-learning With Linear Architectures For Provably Efficient Learning (2022)0.00
• Regularized Q-learning (2022)0.00
• Deep Q-learning: Theoretical Insights From An Asymptotic Analysis (2020)10.35
• A Nearly Optimal And Low-switching Algorithm For Reinforcement Learning With General Function Approximation (2023)0.00
• Easy Monotonic Policy Iteration (2016)0.00
• Beyond Strict Competition: Approximate Convergence Of Multi Agent Q-learning Dynamics (2023)0.00
• Two-timescale Q-learning With Function Approximation In Zero-sum Stochastic Games (2023)0.00
• A Discrete-time Switching System Analysis Of Q-learning (2021)8.35