Learning To Sample With Local And Global Contexts In Experience Replay Buffer

Abstract

Experience replay, which enables the agents to remember and reuse experience from the past, has played a significant role in the success of off-policy reinforcement learning (RL). To utilize the experience replay efficiently, the existing sampling methods allow selecting out more meaningful experiences by imposing priorities on them based on certain metrics (e.g. TD-error). However, they may result in sampling highly biased, redundant transitions since they compute the sampling rate for each transition independently, without consideration of its importance in relation to other transitions. In this paper, we aim to address the issue by proposing a new learning-based sampling method that can compute the relative importance of transition. To this end, we design a novel permutation-equivariant neural architecture that takes contexts from not only features of each transition (local) but also those of others (global) as inputs. We validate our framework, which we refer to as Neural Experienc

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyoh2020learning

Related papers

• Large Batch Experience Replay (2021)0.00
• Introspective Experience Replay: Look Back When Surprised (2022)0.00
• Replay For Safety (2021)0.00
• Prioritized Generative Replay (2024)0.00
• Experience Replay Using Transition Sequences (2017)8.82
• Regret Minimization Experience Replay In Off-policy Reinforcement Learning (2021)0.00
• Neighborhood Mixup Experience Replay: Local Convex Interpolation For Improved Sample Efficiency In Continuous Control Tasks (2022)0.00
• CUER: Corrected Uniform Experience Replay For Off-policy Continuous Deep Reinforcement Learning Algorithms (2024)0.00