Action Candidate Driven Clipped Double Q-learning For Discrete And Continuous Action Tasks

Abstract

Double Q-learning is a popular reinforcement learning algorithm in Markov decision process (MDP) problems. Clipped Double Q-learning, as an effective variant of Double Q-learning, employs the clipped double estimator to approximate the maximum expected action value. Due to the underestimation bias of the clipped double estimator, the performance of clipped Double Q-learning may be degraded in some stochastic environments. In this paper, in order to reduce the underestimation bias, we propose an action candidate-based clipped double estimator for Double Q-learning. Specifically, we first select a set of elite action candidates with high action values from one set of estimators. Then, among these candidates, we choose the highest valued action from the other set of estimators. Finally, we use the maximum value in the second set of estimators to clip the action value of the chosen action in the first set of estimators and the clipped value is used for approximating the maximum expected ac

Stats

• citations25
• S2 citations—
• github stars0
• HF likes0
• heat score10.61
• arxiv keyjiang2022action

Related papers

• Action Candidate Based Clipped Double Q-learning For Discrete And Continuous Action Tasks (2021)0.00
• Exploiting Estimation Bias In Clipped Double Q-learning For Continous Control Reinforcement Learning Tasks (2024)0.00
• Simultaneous Double Q-learning With Conservative Advantage Learning For Actor-critic Methods (2022)0.00
• Finite-time Analysis For Double Q-learning (2020)0.00
• Careful At Estimation And Bold At Exploration (2023)0.00
• Finite-time Analysis Of Simultaneous Double Q-learning (2024)0.00
• Application Of Linear Regression And Quasi-newton Methods To The Deep Reinforcement Learning In Continuous Action Cases (2025)0.00
• On The Estimation Bias In Double Q-learning (2021)0.00