Successor Feature Representations

Abstract

Transfer in Reinforcement Learning aims to improve learning performance on target tasks using knowledge from experienced source tasks. Successor Representations (SR) and their extension Successor Features (SF) are prominent transfer mechanisms in domains where reward functions change between tasks. They reevaluate the expected return of previously learned policies in a new target task to transfer their knowledge. The SF framework extended SR by linearly decomposing rewards into successor features and a reward weight vector allowing their application in high-dimensional tasks. But this came with the cost of having a linear relationship between reward functions and successor features, limiting its application to tasks where such a linear relationship exists. We propose a novel formulation of SR based on learning the cumulative discounted probability of successor features, called Successor Feature Representations (SFR). Crucially, SFR allows to reevaluate the expected return of policies f

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyreinke2021successor

Related papers

• Advantages And Limitations Of Using Successor Features For Transfer In Reinforcement Learning (2017)0.00
• Successor Features For Transfer In Reinforcement Learning (2016)0.00
• SF-DQN: Provable Knowledge Transfer Using Successor Feature For Deep Reinforcement Learning (2024)0.00
• Successor Feature Sets: Generalizing Successor Representations Across Policies (2021)5.84
• Transfer With Model Features In Reinforcement Learning (2018)0.00
• A New Representation Of Successor Features For Transfer Across Dissimilar Environments (2021)0.00
• Universal Successor Representations For Transfer Reinforcement Learning (2018)0.00
• Successor Feature Neural Episodic Control (2021)0.00