Conditional Mutual Information For Disentangled Representations In Reinforcement Learning

Abstract

Reinforcement Learning (RL) environments can produce training data with spurious correlations between features due to the amount of training data or its limited feature coverage. This can lead to RL agents encoding these misleading correlations in their latent representation, preventing the agent from generalising if the correlation changes within the environment or when deployed in the real world. Disentangled representations can improve robustness, but existing disentanglement techniques that minimise mutual information between features require independent features, thus they cannot disentangle correlated features. We propose an auxiliary task for RL algorithms that learns a disentangled representation of high-dimensional observations with correlated features by minimising the conditional mutual information between features in the representation. We demonstrate experimentally, using continuous control tasks, that our approach improves generalisation under correlation shifts, as well

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydunion2023conditional

Related papers

• Temporal Disentanglement Of Representations For Improved Generalisation In Reinforcement Learning (2022)0.00
• Mutual Information Tracks Policy Coherence In Reinforcement Learning (2025)0.00
• Deep Reinforcement Learning With Decorrelation (2019)0.00
• Which Mutual-information Representation Learning Objectives Are Sufficient For Control? (2021)0.00
• Learning And Understanding A Disentangled Feature Representation For Hidden Parameters In Reinforcement Learning (2022)0.00
• Locally Constrained Representations In Reinforcement Learning (2022)0.00
• Gradient Coupling: The Hidden Barrier To Generalization In Agentic Reinforcement Learning (2025)0.00
• Zero-shot Generalization Of Vision-based RL Without Data Augmentation (2024)0.00