Proto Successor Measure: Representing The Behavior Space Of An RL Agent

Abstract

Having explored an environment, intelligent agents should be able to transfer their knowledge to most downstream tasks within that environment without additional interactions. Referred to as "zero-shot learning", this ability remains elusive for general-purpose reinforcement learning algorithms. While recent works have attempted to produce zero-shot RL agents, they make assumptions about the nature of the tasks or the structure of the MDP. We present Proto Successor Measure: the basis set for all possible behaviors of a Reinforcement Learning Agent in a dynamical system. We prove that any possible behavior (represented using visitation distributions) can be represented using an affine combination of these policy-independent basis functions. Given a reward function at test time, we simply need to find the right set of linear weights to combine these bases corresponding to the optimal policy. We derive a practical algorithm to learn these basis functions using reward-free interaction dat

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyagarwal2024proto

Related papers

• Successor Feature Sets: Generalizing Successor Representations Across Policies (2021)5.84
• A First-occupancy Representation For Reinforcement Learning (2021)0.00
• Distributional Successor Features Enable Zero-shot Policy Optimization (2024)0.00
• Count-based Exploration With The Successor Representation (2018)13.17
• Metric Policy Representations For Opponent Modeling (2021)0.00
• Zero-shot Policy Learning With Spatial Temporal Rewarddecomposition On Contingency-aware Observation (2019)0.00
• Good Actions Succeed, Bad Actions Generalize: A Case Study On Why RL Generalizes Better (2025)0.00
• Basis For Intentions: Efficient Inverse Reinforcement Learning Using Past Experience (2022)0.00