Target Transfer Q-learning And Its Convergence Analysis

Abstract

Q-learning is one of the most popular methods in Reinforcement Learning (RL). Transfer Learning aims to utilize the learned knowledge from source tasks to help new tasks to improve the sample complexity of the new tasks. Considering that data collection in RL is both more time and cost consuming and Q-learning converges slowly comparing to supervised learning, different kinds of transfer RL algorithms are designed. However, most of them are heuristic with no theoretical guarantee of the convergence rate. Therefore, it is important for us to clearly understand when and how will transfer learning help RL method and provide the theoretical guarantee for the improvement of the sample complexity. In this paper, we propose to transfer the Q-function learned in the source task to the target of the Q-learning in the new task when certain safe conditions are satisfied. We call this new transfer Q-learning method target transfer Q-Learning. The safe conditions are necessary to avoid the harm to

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2018target

Related papers

• On The Transferability Of Deep-q Networks (2021)0.00
• Transfer Q-learning (2022)0.00
• Regularized Q-learning (2022)0.00
• Target-based Temporal Difference Learning (2019)0.00
• Deep Q-learning: Theoretical Insights From An Asymptotic Analysis (2020)10.35
• Deep Transfer \(q\)-learning For Offline Non-stationary Reinforcement Learning (2025)0.00
• Provable Sim-to-real Transfer In Continuous Domain With Partial Observations (2022)0.00
• Asymptotic Convergence And Performance Of Multi-agent Q-learning Dynamics (2023)0.00