On The Importance Of Feature Decorrelation For Unsupervised Representation Learning In Reinforcement Learning

Abstract

Recently, unsupervised representation learning (URL) has improved the sample efficiency of Reinforcement Learning (RL) by pretraining a model from a large unlabeled dataset. The underlying principle of these methods is to learn temporally predictive representations by predicting future states in the latent space. However, an important challenge of this approach is the representational collapse, where the subspace of the latent representations collapses into a low-dimensional manifold. To address this issue, we propose a novel URL framework that causally predicts future states while increasing the dimension of the latent manifold by decorrelating the features in the latent space. Through extensive empirical studies, we demonstrate that our framework effectively learns predictive representations without collapse, which significantly improves the sample efficiency of state-of-the-art URL methods on the Atari 100k benchmark. The code is available at https://github.com/dojeon-ai/SimTPR.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylee2023on

Code

• dojeon-ai/SimTPR—

Related papers

• Deep Reinforcement Learning With Decorrelation (2019)0.00
• Unsupervised Representation Learning In Deep Reinforcement Learning: A Review (2022)9.59
• Learning And Understanding A Disentangled Feature Representation For Hidden Parameters In Reinforcement Learning (2022)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Data-efficient Reinforcement Learning With Self-predictive Representations (2020)0.00
• POLTER: Policy Trajectory Ensemble Regularization For Unsupervised Reinforcement Learning (2022)0.00
• Locally Constrained Representations In Reinforcement Learning (2022)0.00
• Interpretable Learning Dynamics In Unsupervised Reinforcement Learning (2025)0.00