Temporal Disentanglement Of Representations For Improved Generalisation In Reinforcement Learning

Abstract

Reinforcement Learning (RL) agents are often unable to generalise well to environment variations in the state space that were not observed during training. This issue is especially problematic for image-based RL, where a change in just one variable, such as the background colour, can change many pixels in the image. The changed pixels can lead to drastic changes in the agent's latent representation of the image, causing the learned policy to fail. To learn more robust representations, we introduce TEmporal Disentanglement (TED), a self-supervised auxiliary task that leads to disentangled image representations exploiting the sequential nature of RL observations. We find empirically that RL algorithms utilising TED as an auxiliary task adapt more quickly to changes in environment variables with continued training compared to state-of-the-art representation learning methods. Since TED enforces a disentangled structure of the representation, our experiments also show that policies trained

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydunion2022temporal

Related papers

• Conditional Mutual Information For Disentangled Representations In Reinforcement Learning (2023)0.00
• Zero-shot Generalization Of Vision-based RL Without Data Augmentation (2024)0.00
• Discerning Temporal Difference Learning (2023)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Learning Sparse Representations In Reinforcement Learning (2019)0.00
• Transient Non-stationarity And Generalisation In Deep Reinforcement Learning (2020)0.00
• TEA: Trajectory Encoding Augmentation For Robust And Transferable Policies In Offline Reinforcement Learning (2024)0.00
• Understanding What Affects The Generalization Gap In Visual Reinforcement Learning: Theory And Empirical Evidence (2024)5.84