Value-consistent Representation Learning For Data-efficient Reinforcement Learning

Abstract

Deep reinforcement learning (RL) algorithms suffer severe performance degradation when the interaction data is scarce, which limits their real-world application. Recently, visual representation learning has been shown to be effective and promising for boosting sample efficiency in RL. These methods usually rely on contrastive learning and data augmentation to train a transition model for state prediction, which is different from how the model is used in RL--performing value-based planning. Accordingly, the learned representation by these visual methods may be good for recognition but not optimal for estimating state value and solving the decision problem. To address this issue, we propose a novel method, called value-consistent representation learning (VCR), to learn representations that are directly related to decision-making. More specifically, VCR trains a model to predict the future state (also referred to as the ''imagined state'') based on the current one and a sequence of action

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyue2022value

Related papers

• Accelerating Representation Learning With View-consistent Dynamics In Data-efficient Reinforcement Learning (2022)0.00
• Visual Processing In Context Of Reinforcement Learning (2022)0.00
• The Value-improvement Path: Towards Better Representations For Reinforcement Learning (2020)6.77
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Data-efficient Reinforcement Learning With Self-predictive Representations (2020)0.00
• Value Explicit Pretraining For Learning Transferable Representations (2026)0.00
• Contrastive Learning As Goal-conditioned Reinforcement Learning (2022)0.00
• Harnessing Discrete Representations For Continual Reinforcement Learning (2023)0.00