An Empirical Study On Hyperparameters And Their Interdependence For RL Generalization

Abstract

Recent results in Reinforcement Learning (RL) have shown that agents with limited training environments are susceptible to a large amount of overfitting across many domains. A key challenge for RL generalization is to quantitatively explain the effects of changing parameters on testing performance. Such parameters include architecture, regularization, and RL-dependent variables such as discount factor and action stochasticity. We provide empirical results that show complex and interdependent relationships between hyperparameters and generalization. We further show that several empirical metrics such as gradient cosine similarity and trajectory-dependent metrics serve to provide intuition towards these results.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysong2019an

Related papers

• Hyperparameter Tuning For Deep Reinforcement Learning Applications (2022)0.00
• On The Consistency Of Hyper-parameter Selection In Value-based Deep Reinforcement Learning (2024)0.00
• Measuring And Characterizing Generalization In Deep Reinforcement Learning (2018)9.76
• Quantifying Generalization In Reinforcement Learning (2018)0.00
• Assessing Generalization In Deep Reinforcement Learning (2018)0.00
• A Survey Analyzing Generalization In Deep Reinforcement Learning (2024)0.00
• Dynamics Generalization Via Information Bottleneck In Deep Reinforcement Learning (2020)0.00
• On The Power Of Pre-training For Generalization In RL: Provable Benefits And Hardness (2022)0.00