Temporal Alignment For History Representation In Reinforcement Learning

Abstract

Environments in Reinforcement Learning are usually only partially observable. To address this problem, a possible solution is to provide the agent with information about the past. However, providing complete observations of numerous steps can be excessive. Inspired by human memory, we propose to represent history with only important changes in the environment and, in our approach, to obtain automatically this representation using self-supervision. Our method (TempAl) aligns temporally-close frames, revealing a general, slowly varying state of the environment. This procedure is based on contrastive loss, which pulls embeddings of nearby observations to each other while pushing away other samples from the batch. It can be interpreted as a metric that captures the temporal relations of observations. We propose to combine both common instantaneous and our history representation and we evaluate TempAl on all available Atari games from the Arcade Learning Environment. TempAl surpasses the in

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyermolov2022temporal

Related papers

• Temporal Disentanglement Of Representations For Improved Generalisation In Reinforcement Learning (2022)0.00
• Temporal Representations For Exploration: Learning Complex Exploratory Behavior Without Extrinsic Rewards (2026)0.00
• Temporal Abstractions-augmented Temporally Contrastive Learning: An Alternative To The Laplacian In RL (2022)0.00
• Reconciling Spatial And Temporal Abstractions For Goal Representation (2024)1.69
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Towards Mental Time Travel: A Hierarchical Memory For Reinforcement Learning Agents (2021)0.00
• Learning When To Act: Interval-aware Reinforcement Learning With Predictive Temporal Structure (2026)0.00
• A Temporally Correlated Latent Exploration For Reinforcement Learning (2024)0.00