Quantifying Generalization In Reinforcement Learning

Abstract

In this paper, we investigate the problem of overfitting in deep reinforcement learning. Among the most common benchmarks in RL, it is customary to use the same environments for both training and testing. This practice offers relatively little insight into an agent's ability to generalize. We address this issue by using procedurally generated environments to construct distinct training and test sets. Most notably, we introduce a new environment called CoinRun, designed as a benchmark for generalization in RL. Using CoinRun, we find that agents overfit to surprisingly large training sets. We then show that deeper convolutional architectures improve generalization, as do methods traditionally found in supervised learning, including L2 regularization, dropout, data augmentation and batch normalization.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycobbe2018quantifying

Related papers

• Assessing Generalization In Deep Reinforcement Learning (2018)0.00
• A Survey Analyzing Generalization In Deep Reinforcement Learning (2024)0.00
• The Principle Of Unchanged Optimality In Reinforcement Learning Generalization (2019)0.00
• Measuring And Characterizing Generalization In Deep Reinforcement Learning (2018)9.76
• Illuminating Generalization In Deep Reinforcement Learning Through Procedural Level Generation (2018)0.00
• Improving Generalization In Reinforcement Learning With Mixture Regularization (2020)0.00
• Generalization And Regularization In DQN (2018)0.00
• An Empirical Study On Hyperparameters And Their Interdependence For RL Generalization (2019)0.00