A First-occupancy Representation For Reinforcement Learning

Abstract

Both animals and artificial agents benefit from state representations that support rapid transfer of learning across tasks and which enable them to efficiently traverse their environments to reach rewarding states. The successor representation (SR), which measures the expected cumulative, discounted state occupancy under a fixed policy, enables efficient transfer to different reward structures in an otherwise constant Markovian environment and has been hypothesized to underlie aspects of biological behavior and neural activity. However, in the real world, rewards may move or only be available for consumption once, may shift location, or agents may simply aim to reach goal states as rapidly as possible without the constraint of artificially imposed task horizons. In such cases, the most behaviorally-relevant representation would carry information about when the agent was likely to first reach states of interest, rather than how often it should expect to visit them over a potentially inf

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymoskovitz2021a

Related papers

• Learning Symbolic Representations For Reinforcement Learning Of Non-markovian Behavior (2023)0.00
• A Neurally Plausible Model Learns Successor Representations In Partially Observable Environments (2019)0.00
• A Survey Of State Representation Learning For Deep Reinforcement Learning (2025)0.00
• Proto Successor Measure: Representing The Behavior Space Of An RL Agent (2024)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• State Representation Learning For Control: An Overview (2018)17.39
• Bootstrapped Representations In Reinforcement Learning (2023)0.00
• Successor Feature Sets: Generalizing Successor Representations Across Policies (2021)5.84