Contrastive Learning As Goal-conditioned Reinforcement Learning

Abstract

In reinforcement learning (RL), it is easier to solve a task if given a good representation. While deep RL should automatically acquire such good representations, prior work often finds that learning representations in an end-to-end fashion is unstable and instead equip RL algorithms with additional representation learning parts (e.g., auxiliary losses, data augmentation). How can we design RL algorithms that directly acquire good representations? In this paper, instead of adding representation learning parts to an existing RL algorithm, we show (contrastive) representation learning methods can be cast as RL algorithms in their own right. To do this, we build upon prior work and apply contrastive representation learning to action-labeled trajectories, in such a way that the (inner product of) learned representations exactly corresponds to a goal-conditioned value function. We use this idea to reinterpret a prior RL method as performing contrastive learning, and then use the idea to pro

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyeysenbach2022contrastive

Related papers

• Return-based Contrastive Representation Learning For Reinforcement Learning (2021)12.17
• Contrastive UCB: Provably Efficient Contrastive Self-supervised Learning In Online Reinforcement Learning (2022)0.00
• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• Contrastive Abstraction For Reinforcement Learning (2024)0.00
• Dual Goal Representations (2025)0.00
• TACO: Temporal Latent Action-driven Contrastive Loss For Visual Reinforcement Learning (2023)0.00
• Temporal Abstractions-augmented Temporally Contrastive Learning: An Alternative To The Laplacian In RL (2022)0.00
• Locally Constrained Representations In Reinforcement Learning (2022)0.00