Improving Experience Replay Through Modeling Of Similar Transitions' Sets

Abstract

In this work, we propose and evaluate a new reinforcement learning method, COMPact Experience Replay (COMPER), which uses temporal difference learning with predicted target values based on recurrence over sets of similar transitions, and a new approach for experience replay based on two transitions memories. Our objective is to reduce the required number of experiences to agent training regarding the total accumulated rewarding in the long run. Its relevance to reinforcement learning is related to the small number of observations that it needs to achieve results similar to that obtained by relevant methods in the literature, that generally demand millions of video frames to train an agent on the Atari 2600 games. We report detailed results from five training trials of COMPER for just 100,000 frames and about 25,000 iterations with a small experiences memory on eight challenging games of Arcade Learning Environment (ALE). We also present results for a DQN agent with the same experimenta

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyneves2021improving

Related papers

• Experience Replay Using Transition Sequences (2017)8.82
• Prioritized Sequence Experience Replay (2019)0.00
• Improving Experience Replay With Successor Representation (2021)0.00
• Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23
• Associative Memory Based Experience Replay For Deep Reinforcement Learning (2022)6.34
• Stabilising Experience Replay For Deep Multi-agent Reinforcement Learning (2017)0.00
• Replay For Safety (2021)0.00
• Bootstrapping A DQN Replay Memory With Synthetic Experiences (2020)5.84