Value Explicit Pretraining For Learning Transferable Representations

Abstract

arXiv:2312.12339v3 Announce Type: replace Abstract: Understanding visual inputs for a given task amidst varied changes is a key challenge posed by visual reinforcement learning agents. We propose \textit\{Value Explicit Pretraining\} (VEP), a method that learns generalizable representations for transfer reinforcement learning. VEP enables efficient learning of new tasks that share similar objectives as previously learned tasks, by learning an encoder that trains representations to be invariant to changes in environment dynamics and appearance. To pretrain the encoder with \textit\{suboptimal unlabeled demonstration data\} (sequence of observations and sparse reward signals), we use a self-supervised contrastive loss that enables the model to relate states across different tasks based on the Monte Carlo value estimate that is reflective of task progress, resulting in temporally smooth representations that capture the objective of the task. A major difference between our method and the

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylekkala2026value

Related papers

• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• Contextual Pre-planning On Reward Machine Abstractions For Enhanced Transfer In Deep Reinforcement Learning (2023)2.26
• Pretrain Value, Not Reward: Decoupled Value Policy Optimization (2025)0.00
• Universal Successor Representations For Transfer Reinforcement Learning (2018)0.00
• Rethinking Value Function Learning For Generalization In Reinforcement Learning (2022)0.00
• Towards Principled Representation Learning From Videos For Reinforcement Learning (2024)0.00
• Data-efficient Reinforcement Learning With Self-predictive Representations (2020)0.00
• Dynamic Value Estimation For Single-task Multi-scene Reinforcement Learning (2020)0.00