Contrastive Abstraction For Reinforcement Learning

Abstract

Learning agents with reinforcement learning is difficult when dealing with long trajectories that involve a large number of states. To address these learning problems effectively, the number of states can be reduced by abstract representations that cluster states. In principle, deep reinforcement learning can find abstract states, but end-to-end learning is unstable. We propose contrastive abstraction learning to find abstract states, where we assume that successive states in a trajectory belong to the same abstract state. Such abstract states may be basic locations, achieved subgoals, inventory, or health conditions. Contrastive abstraction learning first constructs clusters of state representations by contrastive learning and then applies modern Hopfield networks to determine the abstract states. The first phase of contrastive abstraction learning is self-supervised learning, where contrastive learning forces states with sequential proximity to have similar representations. The secon

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypatil2024contrastive

Related papers

• Learning Markov State Abstractions For Deep Reinforcement Learning (2021)0.00
• A Theory Of Abstraction In Reinforcement Learning (2022)0.00
• Learning Dynamic Abstract Representations For Sample-efficient Reinforcement Learning (2022)0.00
• Contrastive Learning As Goal-conditioned Reinforcement Learning (2022)0.00
• Discrete State-action Abstraction Via The Successor Representation (2022)0.00
• Summarising And Comparing Agent Dynamics With Contrastive Spatiotemporal Abstraction (2022)0.00
• Temporal Abstractions-augmented Temporally Contrastive Learning: An Alternative To The Laplacian In RL (2022)0.00
• Learning State Abstractions For Transfer In Continuous Control (2020)0.00