Transfer With Model Features In Reinforcement Learning

Abstract

A key question in Reinforcement Learning is which representation an agent can learn to efficiently reuse knowledge between different tasks. Recently the Successor Representation was shown to have empirical benefits for transferring knowledge between tasks with shared transition dynamics. This paper presents Model Features: a feature representation that clusters behaviourally equivalent states and that is equivalent to a Model-Reduction. Further, we present a Successor Feature model which shows that learning Successor Features is equivalent to learning a Model-Reduction. A novel optimization objective is developed and we provide bounds showing that minimizing this objective results in an increasingly improved approximation of a Model-Reduction. Further, we provide transfer experiments on randomly generated MDPs which vary in their transition and reward functions but approximately preserve behavioural equivalence between states. These results demonstrate that Model Features are suitable

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylehnert2018transfer

Related papers

• Advantages And Limitations Of Using Successor Features For Transfer In Reinforcement Learning (2017)0.00
• Successor Features Combine Elements Of Model-free And Model-based Reinforcement Learning (2019)0.00
• A New Representation Of Successor Features For Transfer Across Dissimilar Environments (2021)0.00
• Successor Feature Representations (2021)0.00
• Successor Features For Transfer In Reinforcement Learning (2016)0.00
• Successor Feature Sets: Generalizing Successor Representations Across Policies (2021)5.84
• Self-supervised Reinforcement Learning That Transfers Using Random Features (2023)2.26
• Successor Features For Transfer In Alternating Markov Games (2025)0.00