Deep Reinforcement Learning With Decorrelation

Abstract

Learning an effective representation for high-dimensional data is a challenging problem in reinforcement learning (RL). Deep reinforcement learning (DRL) such as Deep Q networks (DQN) achieves remarkable success in computer games by learning deeply encoded representation from convolution networks. In this paper, we propose a simple yet very effective method for representation learning with DRL algorithms. Our key insight is that features learned by DRL algorithms are highly correlated, which interferes with learning. By adding a regularized loss that penalizes correlation in latent features (with only slight computation), we decorrelate features represented by deep neural networks incrementally. On 49 Atari games, with the same regularization factor, our decorrelation algorithms perform \(70%\) in terms of human-normalized scores, which is \(40%\) better than DQN. In particular, ours performs better than DQN on 39 games with 4 close ties and lost only slightly on \(6\) games. Empirical

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymavrin2019deep

Related papers

• On The Importance Of Feature Decorrelation For Unsupervised Representation Learning In Reinforcement Learning (2023)0.00
• Frustratingly Easy Regularization On Representation Can Boost Deep Reinforcement Learning (2022)0.00
• Generalization And Regularization In DQN (2018)0.00
• Conditional Mutual Information For Disentangled Representations In Reinforcement Learning (2023)0.00
• How Transferable Are The Representations Learned By Deep Q Agents? (2020)0.00
• Learning Sparse Representations Incrementally In Deep Reinforcement Learning (2019)0.00
• Combating Reinforcement Learning's Sisyphean Curse With Intrinsic Fear (2016)0.00
• Unsupervised Representation Learning In Deep Reinforcement Learning: A Review (2022)9.59