Prioritized Sequence Experience Replay

Abstract

Experience replay is widely used in deep reinforcement learning algorithms and allows agents to remember and learn from experiences from the past. In an effort to learn more efficiently, researchers proposed prioritized experience replay (PER) which samples important transitions more frequently. In this paper, we propose Prioritized Sequence Experience Replay (PSER) a framework for prioritizing sequences of experience in an attempt to both learn more efficiently and to obtain better performance. We compare the performance of PER and PSER sampling techniques in a tabular Q-learning environment and in DQN on the Atari 2600 benchmark. We prove theoretically that PSER is guaranteed to converge faster than PER and empirically show PSER substantially improves upon PER.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybrittain2019prioritized

Related papers

• Improving Experience Replay With Successor Representation (2021)0.00
• Associative Memory Based Experience Replay For Deep Reinforcement Learning (2022)6.34
• Reward Prediction Error Prioritisation In Experience Replay: The RPE-PER Method (2025)0.00
• Experience Replay Using Transition Sequences (2017)8.82
• Introspective Experience Replay: Look Back When Surprised (2022)0.00
• Replay For Safety (2021)0.00
• Regret Minimization Experience Replay In Off-policy Reinforcement Learning (2021)0.00
• Large Batch Experience Replay (2021)0.00