Highly Efficient Self-adaptive Reward Shaping For Reinforcement Learning

Abstract

Reward shaping is a technique in reinforcement learning that addresses the sparse-reward problem by providing more frequent and informative rewards. We introduce a self-adaptive and highly efficient reward shaping mechanism that incorporates success rates derived from historical experiences as shaped rewards. The success rates are sampled from Beta distributions, which dynamically evolve from uncertain to reliable values as data accumulates. Initially, the shaped rewards exhibit more randomness to encourage exploration, while over time, the increasing certainty enhances exploitation, naturally balancing exploration and exploitation. Our approach employs Kernel Density Estimation (KDE) combined with Random Fourier Features (RFF) to derive the Beta distributions, providing a computationally efficient, non-parametric, and learning-free solution for high-dimensional continuous state spaces. Our method is validated on various tasks with extremely sparse rewards, demonstrating notable improv

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyma2024highly

Related papers

• Unpacking Reward Shaping: Understanding The Benefits Of Reward Engineering On Sample Complexity (2022)4.52
• BAMDP Shaping: A Unified Framework For Intrinsic Motivation And Reward Shaping (2024)0.00
• Multimodal Reward Shaping For Efficient Exploration In Reinforcement Learning (2021)0.00
• Shaping Sparse Rewards In Reinforcement Learning: A Semi-supervised Approach (2025)0.00
• Learning To Shape Rewards Using A Game Of Two Partners (2021)0.00
• FRESH: Interactive Reward Shaping In High-dimensional State Spaces Using Human Feedback (2020)0.00
• Subgoal-based Reward Shaping To Improve Efficiency In Reinforcement Learning (2021)0.00
• Action Guidance: Getting The Best Of Sparse Rewards And Shaped Rewards For Real-time Strategy Games (2020)0.00