On Hyper-parameter Tuning For Stochastic Optimization Algorithms

Abstract

This paper proposes the first-ever algorithmic framework for tuning hyper-parameters of stochastic optimization algorithm based on reinforcement learning. Hyper-parameters impose significant influences on the performance of stochastic optimization algorithms, such as evolutionary algorithms (EAs) and meta-heuristics. Yet, it is very time-consuming to determine optimal hyper-parameters due to the stochastic nature of these algorithms. We propose to model the tuning procedure as a Markov decision process, and resort the policy gradient algorithm to tune the hyper-parameters. Experiments on tuning stochastic algorithms with different kinds of hyper-parameters (continuous and discrete) for different optimization problems (continuous and discrete) show that the proposed hyper-parameter tuning algorithms do not require much less running times of the stochastic algorithms than bayesian optimization method. The proposed framework can be used as a standard tool for hyper-parameter tuning in sto

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2020on

Related papers

• Online Hyper-parameter Tuning In Off-policy Learning Via Evolutionary Strategies (2020)0.00
• Hyperparameter Tuning For Deep Reinforcement Learning Applications (2022)0.00
• A Framework For History-aware Hyperparameter Optimisation In Reinforcement Learning (2023)0.00
• Automatic Tuning Of Hyper-parameters Of Reinforcement Learning Algorithms Using Bayesian Optimization With Behavioral Cloning (2021)0.00
• A Method For Evaluating Hyperparameter Sensitivity In Reinforcement Learning (2024)0.00
• Learning Optimal Deterministic Policies With Stochastic Policy Gradients (2024)0.00
• Generalized Population-based Training For Hyperparameter Optimization In Reinforcement Learning (2024)9.59
• Hypercontroller: A Hyperparameter Controller For Fast And Stable Training Of Reinforcement Learning Neural Networks (2025)0.00