Stable Hadamard Memory: Revitalizing Memory-augmented Agents For Reinforcement Learning

Abstract

Effective decision-making in partially observable environments demands robust memory management. Despite their success in supervised learning, current deep-learning memory models struggle in reinforcement learning environments that are partially observable and long-term. They fail to efficiently capture relevant past information, adapt flexibly to changing observations, and maintain stable updates over long episodes. We theoretically analyze the limitations of existing memory models within a unified framework and introduce the Stable Hadamard Memory, a novel memory model for reinforcement learning agents. Our model dynamically adjusts memory by erasing no longer needed experiences and reinforcing crucial ones computationally efficiently. To this end, we leverage the Hadamard product for calibrating and updating memory, specifically designed to enhance memory capacity while mitigating numerical and learning challenges. Our approach significantly outperforms state-of-the-art memory-based

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyle2024stable

Related papers

• The Act Of Remembering: A Study In Partially Observable Reinforcement Learning (2020)0.00
• Dynamic Memory For Interpretable Sequential Optimisation (2022)0.00
• Augmented Replay Memory In Reinforcement Learning With Continuous Control (2019)5.24
• Towards Mental Time Travel: A Hierarchical Memory For Reinforcement Learning Agents (2021)0.00
• Influence-aware Memory Architectures For Deep Reinforcement Learning (2019)2.26
• Unsupervised Predictive Memory In A Goal-directed Agent (2018)0.00
• Adamemento: Adaptive Memory-assisted Policy Optimization For Reinforcement Learning (2024)0.00
• How Memory Architecture Affects Learning In A Simple POMDP: The Two-hypothesis Testing Problem (2021)0.00