Generalization In Reinforcement Learning With Selective Noise Injection And Information Bottleneck

Abstract

The ability for policies to generalize to new environments is key to the broad application of RL agents. A promising approach to prevent an agent's policy from overfitting to a limited set of training environments is to apply regularization techniques originally developed for supervised learning. However, there are stark differences between supervised learning and RL. We discuss those differences and propose modifications to existing regularization techniques in order to better adapt them to RL. In particular, we focus on regularization techniques relying on the injection of noise into the learned function, a family that includes some of the most widely used approaches such as Dropout and Batch Normalization. To adapt them to RL, we propose Selective Noise Injection (SNI), which maintains the regularizing effect the injected noise has, while mitigating the adverse effects it has on the gradient quality. Furthermore, we demonstrate that the Information Bottleneck (IB) is a particularly

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyigl2019generalization

Related papers

• Dynamics Generalization Via Information Bottleneck In Deep Reinforcement Learning (2020)0.00
• Regularization Matters In Policy Optimization (2019)2.68
• Emergence Of In-context Reinforcement Learning From Noise Distillation (2023)0.00
• The Principle Of Unchanged Optimality In Reinforcement Learning Generalization (2019)0.00
• Improving Generalization In Reinforcement Learning With Mixture Regularization (2020)0.00
• Quantifying Generalization In Reinforcement Learning (2018)0.00
• Generalization Of Reinforcement Learning With Policy-aware Adversarial Data Augmentation (2021)0.00
• Overestimation, Overfitting, And Plasticity In Actor-critic: The Bitter Lesson Of Reinforcement Learning (2024)0.00