Improving Generalization In Reinforcement Learning With Mixture Regularization

Abstract

Deep reinforcement learning (RL) agents trained in a limited set of environments tend to suffer overfitting and fail to generalize to unseen testing environments. To improve their generalizability, data augmentation approaches (e.g. cutout and random convolution) are previously explored to increase the data diversity. However, we find these approaches only locally perturb the observations regardless of the training environments, showing limited effectiveness on enhancing the data diversity and the generalization performance. In this work, we introduce a simple approach, named mixreg, which trains agents on a mixture of observations from different training environments and imposes linearity constraints on the observation interpolations and the supervision (e.g. associated reward) interpolations. Mixreg increases the data diversity more effectively and helps learn smoother policies. We verify its effectiveness on improving generalization by conducting extensive experiments on the large-s

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2020improving

Related papers

• Generalization Of Reinforcement Learning With Policy-aware Adversarial Data Augmentation (2021)0.00
• Improving Generalization In Meta-rl With Imaginary Tasks From Latent Dynamics Mixture (2021)0.00
• Quantifying Generalization In Reinforcement Learning (2018)0.00
• Gradient Coupling: The Hidden Barrier To Generalization In Agentic Reinforcement Learning (2025)0.00
• Dynamics Generalization Via Information Bottleneck In Deep Reinforcement Learning (2020)0.00
• Generalization In Reinforcement Learning With Selective Noise Injection And Information Bottleneck (2019)0.00
• Assessing Generalization In Deep Reinforcement Learning (2018)0.00
• Regularization Matters In Policy Optimization (2019)2.68