FRESH: Interactive Reward Shaping In High-dimensional State Spaces Using Human Feedback

Abstract

Reinforcement learning has been successful in training autonomous agents to accomplish goals in complex environments. Although this has been adapted to multiple settings, including robotics and computer games, human players often find it easier to obtain higher rewards in some environments than reinforcement learning algorithms. This is especially true of high-dimensional state spaces where the reward obtained by the agent is sparse or extremely delayed. In this paper, we seek to effectively integrate feedback signals supplied by a human operator with deep reinforcement learning algorithms in high-dimensional state spaces. We call this FRESH (Feedback-based REward SHaping). During training, a human operator is presented with trajectories from a replay buffer and then provides feedback on states and actions in the trajectory. In order to generalize feedback signals provided by the human operator to previously unseen states and actions at test-time, we use a feedback neural network. We u

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxiao2020fresh

Related papers

• Highly Efficient Self-adaptive Reward Shaping For Reinforcement Learning (2024)0.00
• Mapping Out The Space Of Human Feedback For Reinforcement Learning: A Conceptual Framework (2024)0.00
• Reward Shaping For Human Learning Via Inverse Reinforcement Learning (2020)0.00
• Subgoal-based Reward Shaping To Improve Efficiency In Reinforcement Learning (2021)0.00
• Deep TAMER: Interactive Agent Shaping In High-dimensional State Spaces (2017)14.73
• Learning Shaping Strategies In Human-in-the-loop Interactive Reinforcement Learning (2018)0.00
• Shaping Advice In Deep Reinforcement Learning (2022)0.00
• Shaping Sparse Rewards In Reinforcement Learning: A Semi-supervised Approach (2025)0.00