MOOSS: Mask-enhanced Temporal Contrastive Learning For Smooth State Evolution In Visual Reinforcement Learning

Abstract

In visual Reinforcement Learning (RL), learning from pixel-based observations poses significant challenges on sample efficiency, primarily due to the complexity of extracting informative state representations from high-dimensional data. Previous methods such as contrastive-based approaches have made strides in improving sample efficiency but fall short in modeling the nuanced evolution of states. To address this, we introduce MOOSS, a novel framework that leverages a temporal contrastive objective with the help of graph-based spatial-temporal masking to explicitly model state evolution in visual RL. Specifically, we propose a self-supervised dual-component strategy that integrates (1) a graph construction of pixel-based observations for spatial-temporal masking, coupled with (2) a multi-level contrastive learning mechanism that enriches state representations by emphasizing temporal continuity and change of states. MOOSS advances the understanding of state dynamics by disrupting and lea

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysun2024mooss

Related papers

• TACO: Temporal Latent Action-driven Contrastive Loss For Visual Reinforcement Learning (2023)0.00
• Temporal Abstractions-augmented Temporally Contrastive Learning: An Alternative To The Laplacian In RL (2022)0.00
• Lifelong Reinforcement Learning With Modulating Masks (2022)0.00
• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• Accelerating Representation Learning With View-consistent Dynamics In Data-efficient Reinforcement Learning (2022)0.00
• Learning Markov State Abstractions For Deep Reinforcement Learning (2021)0.00
• Data-efficient Reinforcement Learning With Self-predictive Representations (2020)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00