A Study On Dense And Sparse (visual) Rewards In Robot Policy Learning

Abstract

Deep Reinforcement Learning (DRL) is a promising approach for teaching robots new behaviour. However, one of its main limitations is the need for carefully hand-coded reward signals by an expert. We argue that it is crucial to automate the reward learning process so that new skills can be taught to robots by their users. To address such automation, we consider task success classifiers using visual observations to estimate the rewards in terms of task success. In this work, we study the performance of multiple state-of-the-art deep reinforcement learning algorithms under different types of reward: Dense, Sparse, Visual Dense, and Visual Sparse rewards. Our experiments in various simulation tasks (Pendulum, Reacher, Pusher, and Fetch Reach) show that while DRL agents can learn successful behaviours using visual rewards when the goal targets are distinguishable, their performance may decrease if the task goal is not clearly visible. Our results also show that visual dense rewards are more

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymohtasib2021a

Related papers

• Multi-objective Model-based Policy Search For Data-efficient Learning With Sparse Rewards (2018)0.00
• Beyond Rewards In Reinforcement Learning For Cyber Defence (2026)0.00
• Viva: Video-trained Value Functions For Guiding Online RL From Diverse Data (2025)0.00
• When Should We Prefer State-to-visual Dagger Over Visual Reinforcement Learning? (2024)0.00
• Learning To Identify Critical States For Reinforcement Learning From Videos (2023)8.76
• Challenges And Opportunities In Offline Reinforcement Learning From Visual Observations (2022)0.00
• Reward Models In Deep Reinforcement Learning: A Survey (2025)0.00
• Reinforcement Learning With Sparse Rewards Using Guidance From Offline Demonstration (2022)0.00