Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning

Abstract

Deep reinforcement learning (RL) agents that exist in high-dimensional state spaces, such as those composed of images, have interconnected learning burdens. Agents must learn an action-selection policy that completes their given task, which requires them to learn a representation of the state space that discerns between useful and useless information. The reward function is the only supervised feedback that RL agents receive, which causes a representation learning bottleneck that can manifest in poor sample efficiency. We present \(k\)-Step Latent (KSL), a new representation learning method that enforces temporal consistency of representations via a self-supervised auxiliary task wherein agents learn to recurrently predict action-conditioned representations of the state space. The state encoder learned by KSL produces low-dimensional representations that make optimization of the RL task more sample efficient. Altogether, KSL produces state-of-the-art results in both data efficiency and

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymcinroe2021learning

Related papers

• Multi-horizon Representations With Hierarchical Forward Models For Reinforcement Learning (2022)0.00
• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• Learning Symbolic Representations For Reinforcement Learning Of Non-markovian Behavior (2023)0.00
• Data-efficient Reinforcement Learning With Self-predictive Representations (2020)0.00
• Iqrl -- Implicitly Quantized Representations For Sample-efficient Reinforcement Learning (2024)0.00
• Simplifying Model-based RL: Learning Representations, Latent-space Models, And Policies With One Objective (2022)0.00
• Bootstrapped Representations In Reinforcement Learning (2023)0.00
• Locally Constrained Representations In Reinforcement Learning (2022)0.00