A Method For Evaluating Hyperparameter Sensitivity In Reinforcement Learning

Abstract

The performance of modern reinforcement learning algorithms critically relies on tuning ever-increasing numbers of hyperparameters. Often, small changes in a hyperparameter can lead to drastic changes in performance, and different environments require very different hyperparameter settings to achieve state-of-the-art performance reported in the literature. We currently lack a scalable and widely accepted approach to characterizing these complex interactions. This work proposes a new empirical methodology for studying, comparing, and quantifying the sensitivity of an algorithm's performance to hyperparameter tuning for a given set of environments. We then demonstrate the utility of this methodology by assessing the hyperparameter sensitivity of several commonly used normalization variants of PPO. The results suggest that several algorithmic performance improvements may, in fact, be a result of an increased reliance on hyperparameter tuning.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyadkins2024a

Related papers

• Generalized Population-based Training For Hyperparameter Optimization In Reinforcement Learning (2024)9.59
• Hyperparameter Tuning For Deep Reinforcement Learning Applications (2022)0.00
• On Hyper-parameter Tuning For Stochastic Optimization Algorithms (2020)0.00
• Hyperparameter Optimisation With Practical Interpretability And Explanation Methods In Probabilistic Curriculum Learning (2025)0.00
• Hyperparameter Optimization Can Even Be Harmful In Off-policy Learning And How To Deal With It (2024)0.00
• On The Consistency Of Hyper-parameter Selection In Value-based Deep Reinforcement Learning (2024)0.00
• Arlbench: Flexible And Efficient Benchmarking For Hyperparameter Optimization In Reinforcement Learning (2024)0.00
• A Framework For History-aware Hyperparameter Optimisation In Reinforcement Learning (2023)0.00