Effective Exploration For Deep Reinforcement Learning Via Bootstrapped Q-ensembles Under Tsallis Entropy Regularization

Abstract

Recently deep reinforcement learning (DRL) has achieved outstanding success on solving many difficult and large-scale RL problems. However the high sample cost required for effective learning often makes DRL unaffordable in resource-limited applications. With the aim of improving sample efficiency and learning performance, we will develop a new DRL algorithm in this paper that seamless integrates entropy-induced and bootstrap-induced techniques for efficient and deep exploration of the learning environment. Specifically, a general form of Tsallis entropy regularizer will be utilized to drive entropy-induced exploration based on efficient approximation of optimal action-selection policies. Different from many existing works that rely on action dithering strategies for exploration, our algorithm is efficient in exploring actions with clear exploration value. Meanwhile, by employing an ensemble of Q-networks under varied Tsallis entropy regularization, the diversity of the ensemble can be

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2018effective

Related papers

• Fast Rates For Maximum Entropy Exploration (2023)0.00
• Maximum Entropy Exploration Without The Rollouts (2026)0.00
• Optimal Scheduling Of Entropy Regulariser For Continuous-time Linear-quadratic Reinforcement Learning (2022)4.52
• Entropy-regularized Diffusion Policy With Q-ensembles For Offline Reinforcement Learning (2024)3.58
• Exploration Versus Exploitation In Reinforcement Learning: A Stochastic Control Approach (2018)9.76
• DQN With Model-based Exploration: Efficient Learning On Environments With Sparse Rewards (2019)0.00
• Sampling Efficient Deep Reinforcement Learning Through Preference-guided Stochastic Exploration (2022)8.09
• Information-directed Exploration For Deep Reinforcement Learning (2018)0.00