Which Rewards Matter? Reward Selection For Reinforcement Learning Under Limited Feedback

Abstract

The ability of reinforcement learning algorithms to learn effective policies is determined by the rewards available during training. However, for practical problems, obtaining large quantities of reward labels is often infeasible due to computational or financial constraints, particularly when relying on human feedback. When reinforcement learning must proceed with limited feedback -- only a fraction of samples get rewards labeled -- a fundamental question arises: which samples should be labeled to maximize policy performance? We formalize this problem of reward selection for reinforcement learning from limited feedback (RLLF), introducing a new problem formulation that facilitates the study of strategies for selecting impactful rewards. Two types of selection strategies are investigated: (i) heuristics that rely on reward-free information such as state visitation and partial value functions, and (ii) strategies pre-trained using auxiliary evaluative feedback. We find that critical sub

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychaudhari2025which

Related papers

• Provably Feedback-efficient Reinforcement Learning Via Active Reward Learning (2023)0.00
• Reward Design For Reinforcement Learning Agents (2025)0.00
• Policy Filtration For RLHF To Mitigate Noise In Reward Models (2024)0.00
• Designing Rewards For Fast Learning (2022)0.00
• Response-level Rewards Are All You Need For Online Reinforcement Learning In Llms: A Mathematical Perspective (2025)0.00
• Replacing Rewards With Examples: Example-based Policy Search Via Recursive Classification (2021)0.00
• Tiered Reward: Designing Rewards For Specification And Fast Learning Of Desired Behavior (2022)0.00
• Reward-conditioned Policies (2019)0.00