Data-efficient Reinforcement Learning With Self-predictive Representations

Abstract

While deep reinforcement learning excels at solving tasks where large amounts of data can be collected through virtually unlimited interaction with the environment, learning from limited interaction remains a key challenge. We posit that an agent can learn more efficiently if we augment reward maximization with self-supervised objectives based on structure in its visual input and sequential interaction with the environment. Our method, Self-Predictive Representations(SPR), trains an agent to predict its own latent state representations multiple steps into the future. We compute target representations for future states using an encoder which is an exponential moving average of the agent's parameters and we make predictions using a learned transition model. On its own, this future prediction objective outperforms prior methods for sample-efficient deep RL from pixels. We further improve performance by adding data augmentation to the future prediction loss, which forces the agent's repres

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyschwarzer2020data

Related papers

• Understanding Self-predictive Learning For Reinforcement Learning (2022)0.00
• Bridging State And History Representations: Understanding Self-predictive RL (2024)0.00
• Ego-foresight: Self-supervised Learning Of Agent-aware Representations For Improved RL (2024)0.00
• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• An Empirical Study On The Power Of Future Prediction In Partially Observable Environments (2024)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Dynamics-aware Embeddings (2019)0.00
• Visual Processing In Context Of Reinforcement Learning (2022)0.00