Learning The Model While Learning Q: Finite-time Sample Complexity Of Online Syncmbq

Abstract

Reinforcement learning has witnessed significant advancements, particularly with the emergence of model-based approaches. Among these, \(Q\)-learning has proven to be a powerful algorithm in model-free settings. However, the extension of \(Q\)-learning to a model-based framework remains relatively unexplored. In this paper, we investigate the sample complexity of \(Q\)-learning when integrated with a model-based approach. The proposed algorihtms learns both the model and Q-value in an online manner. We demonstrate a near-optimal sample complexity result within a broad range of step sizes.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylim2024learning

Related papers

• Is Q-learning Minimax Optimal? A Tight Sample Complexity Analysis (2021)0.00
• Sample Efficient Reinforcement Learning With Partial Dynamics Knowledge (2023)3.58
• Pessimistic Q-learning For Offline Reinforcement Learning: Towards Optimal Sample Complexity (2022)0.00
• Sample Complexity Of Asynchronous Q-learning: Sharper Analysis And Variance Reduction (2020)11.19
• Sample-efficient Reinforcement Learning Is Feasible For Linearly Realizable Mdps With Limited Revisiting (2021)0.00
• Minimax Optimal Q Learning With Nearest Neighbors (2023)8.09
• Sample Complexity Of Variance-reduced Distributionally Robust Q-learning (2023)0.00
• Distributionally Robust Model-based Offline Reinforcement Learning With Near-optimal Sample Complexity (2022)0.00