Fast Task Inference With Variational Intrinsic Successor Features

Abstract

It has been established that diverse behaviors spanning the controllable subspace of an Markov decision process can be trained by rewarding a policy for being distinguishable from other policies \citep\{gregor2016variational, eysenbach2018diversity, warde2018unsupervised\}. However, one limitation of this formulation is generalizing behaviors beyond the finite set being explicitly learned, as is needed for use on subsequent tasks. Successor features \citep\{dayan93improving, barreto2017successor\} provide an appealing solution to this generalization problem, but require defining the reward function as linear in some grounded feature space. In this paper, we show that these two techniques can be combined, and that each method solves the other's primary limitation. To do so we introduce Variational Intrinsic Successor FeatuRes (VISR), a novel algorithm which learns controllable features that can be leveraged to provide enhanced generalization and fast task inference through the successor

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhansen2019fast

Related papers

• Successor Feature Sets: Generalizing Successor Representations Across Policies (2021)5.84
• Universal Successor Features Approximators (2018)0.00
• Successor Features For Transfer In Reinforcement Learning (2016)0.00
• Successor Features Combine Elements Of Model-free And Model-based Reinforcement Learning (2019)0.00
• A New Representation Of Successor Features For Transfer Across Dissimilar Environments (2021)0.00
• Non-adversarial Inverse Reinforcement Learning Via Successor Feature Matching (2024)0.00
• Transfer With Model Features In Reinforcement Learning (2018)0.00
• Successor Feature Neural Episodic Control (2021)0.00