Reward Function Compression Facilitates Goal-dependent Reinforcement Learning

Abstract

Reinforcement learning agents learn from rewards, but humans can uniquely assign value to novel, abstract outcomes in a goal-dependent manner. However, this flexibility is cognitively costly, making learning less efficient. Here, we propose that goal-dependent learning is initially supported by a capacity-limited working memory system. With consistent experience, learners create a "compressed" reward function (a simplified rule defining the goal) which is then transferred to long-term memory and applied automatically upon receiving feedback. This process frees up working memory resources, boosting learning efficiency. We test this theory across six experiments. Consistent with our predictions, our findings demonstrate that learning is parametrically impaired by the size of the goal space, but improves when the goal space structure allows for compression. We also find faster reward processing to correlate with better learning performance, supporting the idea that as goal valuation becom

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymolinaro2025reward

Related papers

• Reward-free Policy Space Compression For Reinforcement Learning (2022)0.00
• Computational Benefits Of Intermediate Rewards For Goal-reaching Policy Learning (2021)0.00
• Reservoir Computing For Fast, Simplified Reinforcement Learning On Memory Tasks (2024)0.00
• Goal-driven Reward By Video Diffusion Models For Reinforcement Learning (2025)0.00
• Generalizing Across Multi-objective Reward Functions In Deep Reinforcement Learning (2018)0.00
• Subgoal-based Reward Shaping To Improve Efficiency In Reinforcement Learning (2021)0.00
• Goal-oriented Inference Of Environment From Redundant Observations (2023)3.58
• Curious Exploration And Return-based Memory Restoration For Deep Reinforcement Learning (2021)0.00