Accelerating Representation Learning With View-consistent Dynamics In Data-efficient Reinforcement Learning

Abstract

Learning informative representations from image-based observations is of fundamental concern in deep Reinforcement Learning (RL). However, data-inefficiency remains a significant barrier to this objective. To overcome this obstacle, we propose to accelerate state representation learning by enforcing view-consistency on the dynamics. Firstly, we introduce a formalism of Multi-view Markov Decision Process (MMDP) that incorporates multiple views of the state. Following the structure of MMDP, our method, View-Consistent Dynamics (VCD), learns state representations by training a view-consistent dynamics model in the latent space, where views are generated by applying data augmentation to states. Empirical evaluation on DeepMind Control Suite and Atari-100k demonstrates VCD to be the SoTA data-efficient algorithm on visual control tasks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhuang2022accelerating

Related papers

• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Improving Sample Efficiency In Model-free Reinforcement Learning From Images (2019)16.99
• Playvirtual: Augmenting Cycle-consistent Virtual Trajectories For Reinforcement Learning (2021)0.00
• Extracting Latent State Representations With Linear Dynamics From Rich Observations (2020)0.00
• Deepmdp: Learning Continuous Latent Space Models For Representation Learning (2019)0.00
• Towards Principled Representation Learning From Videos For Reinforcement Learning (2024)0.00
• Visual Processing In Context Of Reinforcement Learning (2022)0.00