Locally Constrained Representations In Reinforcement Learning

Abstract

The success of Reinforcement Learning (RL) heavily relies on the ability to learn robust representations from the observations of the environment. In most cases, the representations learned purely by the reinforcement learning loss can differ vastly across states depending on how the value functions change. However, the representations learned need not be very specific to the task at hand. Relying only on the RL objective may yield representations that vary greatly across successive time steps. In addition, since the RL loss has a changing target, the representations learned would depend on how good the current values/policies are. Thus, disentangling the representations from the main task would allow them to focus not only on the task-specific features but also the environment dynamics. To this end, we propose locally constrained representations, where an auxiliary loss forces the state representations to be predictable by the representations of the neighboring states. This encourages

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynath2022locally

Related papers

• Bootstrapped Representations In Reinforcement Learning (2023)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• Frustratingly Easy Regularization On Representation Can Boost Deep Reinforcement Learning (2022)0.00
• Return-based Contrastive Representation Learning For Reinforcement Learning (2021)12.17
• Simplifying Model-based RL: Learning Representations, Latent-space Models, And Policies With One Objective (2022)0.00
• On The Generalization Of Representations In Reinforcement Learning (2022)0.00
• State-constrained Offline Reinforcement Learning (2024)0.00