Representation Convergence: Mutual Distillation Is Secretly A Form Of Regularization

Abstract

In this paper, we argue that mutual distillation between reinforcement learning policies serves as an implicit regularization, preventing them from overfitting to irrelevant features. We highlight two separate contributions: (i) Theoretically, for the first time, we prove that enhancing the policy robustness to irrelevant features leads to improved generalization performance. (ii) Empirically, we demonstrate that mutual distillation between policies contributes to such robustness, enabling the spontaneous emergence of invariant representations over pixel inputs. Ultimately, we do not claim to achieve state-of-the-art performance but rather focus on uncovering the underlying principles of generalization and deepening our understanding of its mechanisms.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxie2025representation

Related papers

• How Ensembles Of Distilled Policies Improve Generalisation In Reinforcement Learning (2025)0.00
• Regularization Matters In Policy Optimization (2019)2.68
• DR3: Value-based Deep Reinforcement Learning Requires Explicit Regularization (2021)0.00
• Improving Generalization In Reinforcement Learning With Mixture Regularization (2020)0.00
• Frustratingly Easy Regularization On Representation Can Boost Deep Reinforcement Learning (2022)0.00
• Your Policy Regularizer Is Secretly An Adversary (2022)0.00
• Adversary Agnostic Robust Deep Reinforcement Learning (2020)6.77
• Increasing Entropy To Boost Policy Gradient Performance On Personalization Tasks (2023)0.00