Learning From Visual Observation Via Offline Pretrained State-to-go Transformer

Abstract

Learning from visual observation (LfVO), aiming at recovering policies from only visual observation data, is promising yet a challenging problem. Existing LfVO approaches either only adopt inefficient online learning schemes or require additional task-specific information like goal states, making them not suited for open-ended tasks. To address these issues, we propose a two-stage framework for learning from visual observation. In the first stage, we introduce and pretrain State-to-Go (STG) Transformer offline to predict and differentiate latent transitions of demonstrations. Subsequently, in the second stage, the STG Transformer provides intrinsic rewards for downstream reinforcement learning tasks where an agent learns merely from intrinsic rewards. Empirical results on Atari and Minecraft show that our proposed method outperforms baselines and in some tasks even achieves performance comparable to the policy learned from environmental rewards. These results shed light on the potentia

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhou2023learning

Related papers

• Starformer: Transformer With State-action-reward Representations For Visual Reinforcement Learning (2021)8.82
• Challenges And Opportunities In Offline Reinforcement Learning From Visual Observations (2022)0.00
• Towards Generalisable Imitation Learning Through Conditioned Transition Estimation And Online Behaviour Alignment (2026)0.00
• Learning To Act Without Actions (2023)0.00
• Offline Trajectory Optimization For Offline Reinforcement Learning (2024)1.20
• When Should We Prefer State-to-visual Dagger Over Visual Reinforcement Learning? (2024)0.00
• Lobsdice: Offline Learning From Observation Via Stationary Distribution Correction Estimation (2022)0.00
• Enhancing Vision-language Model Training With Reinforcement Learning In Synthetic Worlds For Real-world Success (2025)0.00