Dynamics-aware Embeddings

Abstract

In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and action sequences. These embeddings capture the structure of the environment's dynamics, enabling efficient policy learning. We demonstrate that our action embeddings alone improve the sample efficiency and peak performance of model-free RL on control from low-dimensional states. By combining state and action embeddings, we achieve efficient learning of high-quality policies on goal-conditioned continuous control from pixel observations in only 1-2 million environment steps.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywhitney2019dynamics

Related papers

• Learn Dynamic-aware State Embedding For Transfer Learning (2021)0.00
• Data-efficient Reinforcement Learning With Self-predictive Representations (2020)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Embed To Control Partially Observed Systems: Representation Learning With Provable Sample Efficiency (2022)0.00
• Iqrl -- Implicitly Quantized Representations For Sample-efficient Reinforcement Learning (2024)0.00
• Understanding Self-predictive Learning For Reinforcement Learning (2022)0.00
• Contrastive Behavioral Similarity Embeddings For Generalization In Reinforcement Learning (2021)0.00
• Ego-foresight: Self-supervised Learning Of Agent-aware Representations For Improved RL (2024)0.00