Autolr: Layer-wise Pruning And Auto-tuning Of Learning Rates In Fine-tuning Of Deep Networks

Abstract

Existing fine-tuning methods use a single learning rate over all layers. In this paper, first, we discuss that trends of layer-wise weight variations by fine-tuning using a single learning rate do not match the well-known notion that lower-level layers extract general features and higher-level layers extract specific features. Based on our discussion, we propose an algorithm that improves fine-tuning performance and reduces network complexity through layer-wise pruning and auto-tuning of layer-wise learning rates. The proposed algorithm has verified the effectiveness by achieving state-of-the-art performance on the image retrieval benchmark datasets (CUB-200, Cars-196, Stanford online product, and Inshop). Code is available at https://github.com/youngminPIL/AutoLR.

Stats

• citations0
• S2 citations—
• github stars17
• HF likes0
• heat score2.51
• arxiv keyro2020autolr

Code

• youngminPIL/AutoLR—

Related papers

• Progressive Local Filter Pruning For Image Retrieval Acceleration (2020)12.74
• Pruning Convolutional Neural Networks For Image Instance Retrieval (2017)0.00
• OPERA: Online Data Pruning For Efficient Retrieval Model Adaptation (2026)0.00
• Grafit: Learning Fine-grained Image Representations With Coarse Labels (2020)12.93
• Making Large Language Models Efficient Dense Retrievers (2025)0.00
• Coarse-to-fine: Learning Compact Discriminative Representation For Single-stage Image Retrieval (2023)9.35
• Infusing Fine-grained Visual Knowledge To Vision-language Models (2025)0.00
• Mildnet: A Lightweight Single Scaled Deep Ranking Architecture (2019)0.00