Langevin DQN

Abstract

Algorithms that tackle deep exploration -- an important challenge in reinforcement learning -- have relied on epistemic uncertainty representation through ensembles or other hypermodels, exploration bonuses, or visitation count distributions. An open question is whether deep exploration can be achieved by an incremental reinforcement learning algorithm that tracks a single point estimate, without additional complexity required to account for epistemic uncertainty. We answer this question in the affirmative. In particular, we develop Langevin DQN, a variation of DQN that differs only in perturbing parameter updates with Gaussian noise and demonstrate through a computational study that the presented algorithm achieves deep exploration. We also offer some intuition to how Langevin DQN achieves deep exploration. In addition, we present a modification of the Langevin DQN algorithm to improve the computational efficiency.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydwaracherla2020langevin

Related papers

• DQN With Model-based Exploration: Efficient Learning On Environments With Sparse Rewards (2019)0.00
• Information-directed Exploration For Deep Reinforcement Learning (2018)0.00
• The Uncertainty Bellman Equation And Exploration (2017)0.00
• On The Convergence And Sample Complexity Analysis Of Deep Q-networks With \(\epsilon\)-greedy Exploration (2023)3.58
• Sampling Efficient Deep Reinforcement Learning Through Preference-guided Stochastic Exploration (2022)8.09
• Neighboring State-based Exploration For Reinforcement Learning (2022)0.00
• Interpretable Option Discovery Using Deep Q-learning And Variational Autoencoders (2022)0.00
• \(\beta\)-dqn: Improving Deep Q-learning By Evolving The Behavior (2025)0.00