Deep Transfer \(q\)-learning For Offline Non-stationary Reinforcement Learning

Abstract

In dynamic decision-making scenarios across business and healthcare, leveraging sample trajectories from diverse populations can significantly enhance reinforcement learning (RL) performance for specific target populations, especially when sample sizes are limited. While existing transfer learning methods primarily focus on linear regression settings, they lack direct applicability to reinforcement learning algorithms. This paper pioneers the study of transfer learning for dynamic decision scenarios modeled by non-stationary finite-horizon Markov decision processes, utilizing neural networks as powerful function approximators and backward inductive learning. We demonstrate that naive sample pooling strategies, effective in regression settings, fail in Markov decision processes.To address this challenge, we introduce a novel ``re-weighted targeting procedure'' to construct ``transferable RL samples'' and propose ``transfer deep \(Q^*\)-learning'', enabling neural network approximation w

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychai2025deep

Related papers

• Transfer Q-learning (2022)0.00
• On The Transferability Of Deep-q Networks (2021)0.00
• Adaptive \(q\)-network: On-the-fly Target Selection For Deep Reinforcement Learning (2024)0.00
• Q-learning Decision Transformer: Leveraging Dynamic Programming For Conditional Sequence Modelling In Offline RL (2022)0.00
• Trajectory Data Suffices For Statistically Efficient Learning In Offline RL With Linear \(q^\pi\)-realizability And Concentrability (2024)0.00
• Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency From Shifted-dynamics Data (2024)0.00
• Q-value Regularized Decision Convformer For Offline Reinforcement Learning (2024)0.00
• Boosting Offline Reinforcement Learning With Residual Generative Modeling (2021)0.00