Temporal Abstractions-augmented Temporally Contrastive Learning: An Alternative To The Laplacian In RL

Abstract

In reinforcement learning, the graph Laplacian has proved to be a valuable tool in the task-agnostic setting, with applications ranging from skill discovery to reward shaping. Recently, learning the Laplacian representation has been framed as the optimization of a temporally-contrastive objective to overcome its computational limitations in large (or continuous) state spaces. However, this approach requires uniform access to all states in the state space, overlooking the exploration problem that emerges during the representation learning process. In this work, we propose an alternative method that is able to recover, in a non-uniform-prior setting, the expressiveness and the desired properties of the Laplacian representation. We do so by combining the representation learning with a skill-based covering policy, which provides a better training distribution to extend and refine the representation. We also show that a simple augmentation of the representation objective with the learned te

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyerraqabi2022temporal

Related papers

• Proper Laplacian Representation Learning (2023)0.00
• TACO: Temporal Latent Action-driven Contrastive Loss For Visual Reinforcement Learning (2023)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Contrastive Abstraction For Reinforcement Learning (2024)0.00
• Contrastive Learning As Goal-conditioned Reinforcement Learning (2022)0.00
• Multi-timescale, Gradient Descent, Temporal Difference Learning With Linear Options (2017)0.00
• Temporal Disentanglement Of Representations For Improved Generalisation In Reinforcement Learning (2022)0.00
• Temporal Representations For Exploration: Learning Complex Exploratory Behavior Without Extrinsic Rewards (2026)0.00