Abstract
Can gradient descent be accelerated by just choosing better stepsizes? Surprisingly, the answer is yes. This short expository article provides an accessible introduction to this phenomenon of stepsize hedging.
Stepsize Hedging: an Alternative Mechanism for Accelerating Gradient Descent
Related papers
Ranked by semantic similarity β how closely each paper's abstract matches this one (100% = near-identical topic).
- Acceleration by Stepsize Hedging I: Multi-Step Descent and the Silver Stepsize Schedule (2023)71% match
- First-ish Order Methods: Hessian-aware Scalings of Gradient Descent (2025)67% match
- Safeguarding adaptive methods: global convergence of Barzilai-Borwein and other stepsize choices (2024)66% match
- Step-Size Stability in Stochastic Optimization: A Theoretical Perspective (2026)64% match
- Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent (2025)63% match
- SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance (2023)63% match
- Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression (2025)62% match
- Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD (2026)62% match