Can A MISL Fly? Analysis And Ingredients For Mutual Information Skill Learning

Abstract

Self-supervised learning has the potential of lifting several of the key challenges in reinforcement learning today, such as exploration, representation learning, and reward design. Recent work (METRA) has effectively argued that moving away from mutual information and instead optimizing a certain Wasserstein distance is important for good performance. In this paper, we argue that the benefits seen in that paper can largely be explained within the existing framework of mutual information skill learning (MISL). Our analysis suggests a new MISL method (contrastive successor features) that retains the excellent performance of METRA with fewer moving parts, and highlights connections between skill learning, contrastive representation learning, and successor features. Finally, through careful ablation studies, we provide further insight into some of the key ingredients for both our method and METRA.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzheng2024can

Related papers

• Skill-aware Mutual Information Optimisation For Generalisation In Reinforcement Learning (2024)0.00
• Self-improving Skill Learning For Robust Skill-based Meta-reinforcement Learning (2025)0.00
• Mutual Information Tracks Policy Coherence In Reinforcement Learning (2025)0.00
• Which Mutual-information Representation Learning Objectives Are Sufficient For Control? (2021)0.00
• Intrinsically Motivated Self-supervised Learning In Reinforcement Learning (2021)3.58
• PMIC: Improving Multi-agent Reinforcement Learning With Progressive Mutual Information Collaboration (2022)0.00
• Robust Multi-agent Reinforcement Learning By Mutual Information Regularization (2023)0.00
• Mutual Information Regularized Offline Reinforcement Learning (2022)0.00