Successor Feature Sets: Generalizing Successor Representations Across Policies

Abstract

Successor-style representations have many advantages for reinforcement learning: for example, they can help an agent generalize from past experience to new goals, and they have been proposed as explanations of behavioral and neural data from human and animal learners. They also form a natural bridge between model-based and model-free RL methods: like the former they make predictions about future experiences, and like the latter they allow efficient prediction of total discounted rewards. However, successor-style representations are not optimized to generalize across policies: typically, we maintain a limited-length list of policies, and share information among them by representation learning or GPI. Successor-style representations also typically make no provision for gathering information or reasoning about latent variables. To address these limitations, we bring together ideas from predictive state representations, belief space value iteration, successor features, and convex analysis:

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keybrantley2021successor

Related papers

• Successor Features Combine Elements Of Model-free And Model-based Reinforcement Learning (2019)0.00
• Successor Feature Representations (2021)0.00
• Advantages And Limitations Of Using Successor Features For Transfer In Reinforcement Learning (2017)0.00
• A New Representation Of Successor Features For Transfer Across Dissimilar Environments (2021)0.00
• Transfer With Model Features In Reinforcement Learning (2018)0.00
• Distributional Successor Features Enable Zero-shot Policy Optimization (2024)0.00
• Universal Successor Features Approximators (2018)0.00
• Successor Features For Transfer In Reinforcement Learning (2016)0.00