Mitigating Estimation Bias With Representation Learning In TD Error-driven Regularization

Abstract

Deterministic policy gradient algorithms for continuous control suffer from value estimation biases that degrade performance. While double critics reduce such biases, the exploration potential of double actors remains underexplored. Building on temporal-difference error-driven regularization (TDDR), a double actor-critic framework, this work introduces enhanced methods to achieve flexible bias control and stronger representation learning. We propose three convex combination strategies, symmetric and asymmetric, that balance pessimistic estimates to mitigate overestimation and optimistic exploration via double actors to alleviate underestimation. A single hyperparameter governs this mechanism, enabling tunable control across the bias spectrum. To further improve performance, we integrate augmented state and action representations into the actor and critic networks. Extensive experiments show that our approach consistently outperforms benchmarks, demonstrating the value of tunable bias a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2025mitigating

Related papers

• Double Actor-critic With TD Error-driven Regularization In Reinforcement Learning (2024)3.58
• Mitigating Estimation Errors By Twin Td-regularized Actor And Critic For Deep Reinforcement Learning (2023)0.00
• Softmax Deep Double Deterministic Policy Gradients (2020)0.00
• Exploiting Estimation Bias In Clipped Double Q-learning For Continous Control Reinforcement Learning Tasks (2024)0.00
• Value Activation For Bias Alleviation: Generalized-activated Deep Double Deterministic Policy Gradients (2021)0.00
• Moderate Actor-critic Methods: Controlling Overestimation Bias Via Expectile Loss (2025)0.00
• Ader:adapting Between Exploration And Robustness For Actor-critic Methods (2021)0.00
• Parameter-free Reduction Of The Estimation Bias In Deep Reinforcement Learning For Deterministic Policy Gradients (2021)0.00