Dynamics Generalization Via Information Bottleneck In Deep Reinforcement Learning

Abstract

Despite the significant progress of deep reinforcement learning (RL) in solving sequential decision making problems, RL agents often overfit to training environments and struggle to adapt to new, unseen environments. This prevents robust applications of RL in real world situations, where system dynamics may deviate wildly from the training settings. In this work, our primary contribution is to propose an information theoretic regularization objective and an annealing-based optimization method to achieve better generalization ability in RL agents. We demonstrate the extreme generalization benefits of our approach in different domains ranging from maze navigation to robotic tasks; for the first time, we show that agents can generalize to test parameters more than 10 standard deviations away from the training parameter distribution. This work provides a principled way to improve generalization in RL by gradually removing information that is redundant for task-solving; it opens doors for t

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylu2020dynamics

Related papers

• Generalization In Reinforcement Learning With Selective Noise Injection And Information Bottleneck (2019)0.00
• Assessing Generalization In Deep Reinforcement Learning (2018)0.00
• Illuminating Generalization In Deep Reinforcement Learning Through Procedural Level Generation (2018)0.00
• Generalization Through The Lens Of Learning Dynamics (2022)0.00
• The Principle Of Unchanged Optimality In Reinforcement Learning Generalization (2019)0.00
• Measuring And Characterizing Generalization In Deep Reinforcement Learning (2018)9.76
• Improving Performance In Reinforcement Learning By Breaking Generalization In Neural Networks (2020)0.00
• Gradient Coupling: The Hidden Barrier To Generalization In Agentic Reinforcement Learning (2025)0.00