Low-pass Recurrent Neural Networks - A Memory Architecture For Longer-term Correlation Discovery

Abstract

Reinforcement learning (RL) agents performing complex tasks must be able to remember observations and actions across sizable time intervals. This is especially true during the initial learning stages, when exploratory behaviour can increase the delay between specific actions and their effects. Many new or popular approaches for learning these distant correlations employ backpropagation through time (BPTT), but this technique requires storing observation traces long enough to span the interval between cause and effect. Besides memory demands, learning dynamics like vanishing gradients and slow convergence due to infrequent weight updates can reduce BPTT's practicality; meanwhile, although online recurrent network learning is a developing topic, most approaches are not efficient enough to use as replacements. We propose a simple, effective memory strategy that can extend the window over which BPTT can learn without requiring longer traces. We explore this approach empirically on a few ta

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keystepleton2018low

Related papers

• Deep Reinforcement Learning With Time-scale Invariant Memory (2024)0.00
• Reservoir Computing For Fast, Simplified Reinforcement Learning On Memory Tasks (2024)0.00
• Influence-aware Memory Architectures For Deep Reinforcement Learning (2019)2.26
• Perception-prediction-reaction Agents For Deep Reinforcement Learning (2020)0.00
• Map-based Experience Replay: A Memory-efficient Solution To Catastrophic Forgetting In Reinforcement Learning (2023)9.23
• Recurrent Action Transformer With Memory (2023)0.00
• Augmented Replay Memory In Reinforcement Learning With Continuous Control (2019)5.24
• Generative Memory For Lifelong Reinforcement Learning (2019)0.00