Lucid Dreaming For Experience Replay: Refreshing Past States With The Current Policy
2020 · Yunshu Du, Garrett Warnell, Assefaw Gebremedhin, et al.
Abstract
Experience replay (ER) improves the data efficiency of off-policy reinforcement learning (RL) algorithms by allowing an agent to store and reuse its past experiences in a replay buffer. While many techniques have been proposed to enhance ER by biasing how experiences are sampled from the buffer, thus far they have not considered strategies for refreshing experiences inside the buffer. In this work, we introduce Lucid Dreaming for Experience Replay (LiDER), a conceptually new framework that allows replay experiences to be refreshed by leveraging the agent's current policy. LiDER consists of three steps: First, LiDER moves an agent back to a past state. Second, from that state, LiDER then lets the agent execute a sequence of actions by following its current policy -- as if the agent were "dreaming" about the past and can try out different behaviors to encounter new experiences in the dream. Third, LiDER stores and reuses the new experience if it turned out better than what the agent prev
Authors
(none)
Tags
Stats
Related papers
- Remember And Forget For Experience Replay (2018)0.00
- Replay For Safety (2021)0.00
- Introspective Experience Replay: Look Back When Surprised (2022)0.00
- CUER: Corrected Uniform Experience Replay For Off-policy Continuous Deep Reinforcement Learning Algorithms (2024)0.00
- Improvements Of Dark Experience Replay And Reservoir Sampling Towards Better Balance Between Consolidation And Plasticity (2025)0.00
- Replay-enhanced Continual Reinforcement Learning (2023)0.00
- Frugal Actor-critic: Sample Efficient Off-policy Deep Reinforcement Learning Using Unique Experiences (2024)0.00
- Accmer: Accelerating Multi-agent Experience Replay With Cache Locality-aware Prioritization (2023)5.24