Smooth Q-learning: Accelerate Convergence Of Q-learning Using Similarity

Abstract

An improvement of Q-learning is proposed in this paper. It is different from classic Q-learning in that the similarity between different states and actions is considered in the proposed method. During the training, a new updating mechanism is used, in which the Q value of the similar state-action pairs are updated synchronously. The proposed method can be used in combination with both tabular Q-learning function and deep Q-learning. And the results of numerical examples illustrate that compared to the classic Q-learning, the proposed method has a significantly better performance.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyliao2021smooth

Related papers

• The Mean-squared Error Of Double Q-learning (2020)0.00
• Enhancing Q-value Updates In Deep Q-learning Via Successor-state Prediction (2025)0.00
• SAVGO: Learning State-action Value Geometry With Cosine Similarity For Continuous Control (2026)0.00
• Enhancing Control Policy Smoothness By Aligning Actions With Predictions From Preceding States (2026)0.00
• Regularized Q-learning (2022)0.00
• How To Discretize Continuous State-action Spaces In Q-learning: A Symbolic Control Approach (2024)3.58
• Symmetric Q-learning: Reducing Skewness Of Bellman Error In Online Reinforcement Learning (2024)0.00
• Q-learning As A Monotone Scheme (2024)0.00