R\'enyi State Entropy For Exploration Acceleration In Reinforcement Learning

Abstract

One of the most critical challenges in deep reinforcement learning is to maintain the long-term exploration capability of the agent. To tackle this problem, it has been recently proposed to provide intrinsic rewards for the agent to encourage exploration. However, most existing intrinsic reward-based methods proposed in the literature fail to provide sustainable exploration incentives, a problem known as vanishing rewards. In addition, these conventional methods incur complex models and additional memory in their learning procedures, resulting in high computational complexity and low robustness. In this work, a novel intrinsic reward module based on the R\'enyi entropy is proposed to provide high-quality intrinsic rewards. It is shown that the proposed method actually generalizes the existing state entropy maximization methods. In particular, a \(k\)-nearest neighbor estimator is introduced for entropy estimation while a \(k\)-value search method is designed to guarantee the estimation

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyuan2022r

Related papers

• Maximum Entropy Exploration Without The Rollouts (2026)0.00
• Accelerating Reinforcement Learning With Value-conditional State Entropy Exploration (2023)0.00
• Maximum-entropy Exploration With Future State-action Visitation Measures (2026)0.00
• Multimodal Reward Shaping For Efficient Exploration In Reinforcement Learning (2021)0.00
• Information Content Exploration (2023)0.00
• Fast Rates For Maximum Entropy Exploration (2023)0.00
• Off-policy Maximum Entropy RL With Future State And Action Visitation Measures (2024)0.00
• Intrinsic Rewards For Exploration Without Harm From Observational Noise: A Simulation Study Based On The Free Energy Principle (2024)0.00