Successor Features Combine Elements Of Model-free And Model-based Reinforcement Learning

Abstract

A key question in reinforcement learning is how an intelligent agent can generalize knowledge across different inputs. By generalizing across different inputs, information learned for one input can be immediately reused for improving predictions for another input. Reusing information allows an agent to compute an optimal decision-making strategy using less data. State representation is a key element of the generalization process, compressing a high-dimensional input space into a low-dimensional latent state space. This article analyzes properties of different latent state spaces, leading to new connections between model-based and model-free reinforcement learning. Successor features, which predict frequencies of future observations, form a link between model-based and model-free learning: Learning to predict future expected reward outcomes, a key characteristic of model-based agents, is equivalent to learning successor features. Learning successor features is a form of temporal differe

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylehnert2019successor

Related papers

• Successor Feature Sets: Generalizing Successor Representations Across Policies (2021)5.84
• Transfer With Model Features In Reinforcement Learning (2018)0.00
• Advantages And Limitations Of Using Successor Features For Transfer In Reinforcement Learning (2017)0.00
• Successor Feature Neural Episodic Control (2021)0.00
• Successor Feature Representations (2021)0.00
• A Neurally Plausible Model Learns Successor Representations In Partially Observable Environments (2019)0.00
• A New Representation Of Successor Features For Transfer Across Dissimilar Environments (2021)0.00
• Successor Features For Transfer In Reinforcement Learning (2016)0.00