SF-DQN: Provable Knowledge Transfer Using Successor Feature For Deep Reinforcement Learning

Abstract

This paper studies the transfer reinforcement learning (RL) problem where multiple RL problems have different reward functions but share the same underlying transition dynamics. In this setting, the Q-function of each RL problem (task) can be decomposed into a successor feature (SF) and a reward mapping: the former characterizes the transition dynamics, and the latter characterizes the task-specific reward function. This Q-function decomposition, coupled with a policy improvement operator known as generalized policy improvement (GPI), reduces the sample complexity of finding the optimal Q-function, and thus the SF \& GPI framework exhibits promising empirical performance compared to traditional RL methods like Q-learning. However, its theoretical foundations remain largely unestablished, especially when learning the successor features using deep neural networks (SF-DQN). This paper studies the provable knowledge transfer using SFs-DQN in transfer RL problems. We establish the first con

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2024sf

Related papers

• Successor Feature Representations (2021)0.00
• Successor Features For Transfer In Reinforcement Learning (2016)0.00
• A New Representation Of Successor Features For Transfer Across Dissimilar Environments (2021)0.00
• Uncertainty-aware Transfer Across Tasks Using Hybrid Model-based Successor Feature Reinforcement Learning (2023)5.24
• Advantages And Limitations Of Using Successor Features For Transfer In Reinforcement Learning (2017)0.00
• Successor Features For Transfer In Alternating Markov Games (2025)0.00
• On The Transferability Of Deep-q Networks (2021)0.00
• Successor Feature Neural Episodic Control (2021)0.00