Double Q(\(\sigma\)) And Q(\(\sigma, \lambda\)): Unifying Reinforcement Learning Control Algorithms

Abstract

Temporal-difference (TD) learning is an important field in reinforcement learning. Sarsa and Q-Learning are among the most used TD algorithms. The Q(\(\sigma\)) algorithm (Sutton and Barto (2017)) unifies both. This paper extends the Q(\(\sigma\)) algorithm to an online multi-step algorithm Q(\(\sigma, \lambda\)) using eligibility traces and introduces Double Q(\(\sigma\)) as the extension of Q(\(\sigma\)) to double learning. Experiments suggest that the new Q(\(\sigma, \lambda\)) algorithm can outperform the classical TD control methods Sarsa(\(\lambda\)), Q(\(\lambda\)) and Q(\(\sigma\)).

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydumke2017double

Related papers

• Multi-step Reinforcement Learning: A Unifying Algorithm (2017)12.68
• A Unified Approach For Multi-step Temporal-difference Learning With Eligibility Traces In Reinforcement Learning (2018)6.77
• Finite-time Analysis Of Simultaneous Double Q-learning (2024)0.00
• An Analysis Of Quantile Temporal-difference Learning (2023)0.00
• Finite-time Analysis For Double Q-learning (2020)0.00
• Simplifying Deep Temporal Difference Learning (2024)0.00
• Structured Difference-of-q Via Orthogonal Learning (2024)0.00
• Simultaneous Double Q-learning With Conservative Advantage Learning For Actor-critic Methods (2022)0.00