On The Consistency Of Hyper-parameter Selection In Value-based Deep Reinforcement Learning

Abstract

Deep reinforcement learning (deep RL) has achieved tremendous success on various domains through a combination of algorithmic design and careful selection of hyper-parameters. Algorithmic improvements are often the result of iterative enhancements built upon prior approaches, while hyper-parameter choices are typically inherited from previous methods or fine-tuned specifically for the proposed technique. Despite their crucial impact on performance, hyper-parameter choices are frequently overshadowed by algorithmic advancements. This paper conducts an extensive empirical study focusing on the reliability of hyper-parameter selection for value-based deep reinforcement learning agents, including the introduction of a new score to quantify the consistency and reliability of various hyper-parameters. Our findings not only help establish which hyper-parameters are most critical to tune, but also help clarify which tunings remain consistent across different training regimes.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyobandoceron2024on

Related papers

• Hyperparameter Tuning For Deep Reinforcement Learning Applications (2022)0.00
• An Empirical Study On Hyperparameters And Their Interdependence For RL Generalization (2019)0.00
• A Framework For History-aware Hyperparameter Optimisation In Reinforcement Learning (2023)0.00
• Towards Hyperparameter-free Policy Selection For Offline Reinforcement Learning (2021)0.00
• A Method For Evaluating Hyperparameter Sensitivity In Reinforcement Learning (2024)0.00
• Dissecting Deep RL With High Update Ratios: Combatting Value Divergence (2024)0.00
• Automatic Tuning Of Hyper-parameters Of Reinforcement Learning Algorithms Using Bayesian Optimization With Behavioral Cloning (2021)0.00
• What Matters In On-policy Reinforcement Learning? A Large-scale Empirical Study (2020)0.00