Learning-driven Exploration For Reinforcement Learning

Abstract

Effective and intelligent exploration has been an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies such as \(\epsilon\)-greedy exploration or adding Gaussian noise to actions. These heuristics, however, are unable to intelligently distinguish the well explored and the unexplored regions of state space, which can lead to inefficient use of training time. We introduce entropy-based exploration (EBE) that enables an agent to explore efficiently the unexplored regions of state space. EBE quantifies the agent's learning in a state using merely state-dependent action values and adaptively explores the state space, i.e. more exploration for the unexplored region of the state space. We perform experiments on a diverse set of environments and demonstrate that EBE enables efficient exploration that ultimately results in faster learning without having to tune any hyperparameter. The code to reproduce the experiments is

Stats

• citations4
• S2 citations—
• github stars3
• HF likes0
• heat score6.45
• arxiv keyusama2019learning

Related papers

• Exploration Via Elliptical Episodic Bonuses (2022)3.58
• Random Latent Exploration For Deep Reinforcement Learning (2024)0.00
• Accelerating Reinforcement Learning With Value-conditional State Entropy Exploration (2023)0.00
• Neighboring State-based Exploration For Reinforcement Learning (2022)0.00
• Maximum Entropy Exploration Without The Rollouts (2026)0.00
• Fast Rates For Maximum Entropy Exploration (2023)0.00
• Behavioral Entropy-guided Dataset Generation For Offline Reinforcement Learning (2025)0.00
• R\'enyi State Entropy For Exploration Acceleration In Reinforcement Learning (2022)0.00