Recurrent Action Transformer With Memory

Abstract

Transformers have become increasingly popular in offline reinforcement learning (RL) due to their ability to treat agent trajectories as sequences, reframing policy learning as a sequence modeling task. However, in partially observable environments (POMDPs), effective decision-making depends on retaining information about past events -- something that standard transformers struggle with due to the quadratic complexity of self-attention, which limits their context length. One solution to this problem is to extend transformers with memory mechanisms. We propose the Recurrent Action Transformer with Memory (RATE), a novel transformer-based architecture for offline RL that incorporates a recurrent memory mechanism designed to regulate information retention. We evaluate RATE across a diverse set of environments: memory-intensive tasks (ViZDoom-Two-Colors, T-Maze, Memory Maze, Minigrid-Memory, and POPGym), as well as standard Atari and MuJoCo benchmarks. Our comprehensive experiments demonst

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycherepanov2023recurrent

Related papers

• From Memories To Maps: Mechanisms Of In-context Reinforcement Learning In Transformers (2025)0.00
• P2DT: Mitigating Forgetting In Task-incremental Learning With Progressive Prompt Decision Transformer (2024)3.58
• Low-pass Recurrent Neural Networks - A Memory Architecture For Longer-term Correlation Discovery (2018)0.00
• Return-aligned Decision Transformer (2024)1.69
• Decision Mamba: A Multi-grained State Space Model With Self-evolution Regularization For Offline RL (2024)0.00
• The Act Of Remembering: A Study In Partially Observable Reinforcement Learning (2020)0.00
• Solving Continual Offline Reinforcement Learning With Decision Transformer (2024)0.00
• Harmodt: Harmony Multi-task Decision Transformer For Offline Reinforcement Learning (2024)0.00