Darkrank: Accelerating Deep Metric Learning Via Cross Sample Similarities Transfer

Abstract

We have witnessed rapid evolution of deep neural network architecture design in the past years. These latest progresses greatly facilitate the developments in various areas such as computer vision and natural language processing. However, along with the extraordinary performance, these state-of-the-art models also bring in expensive computational cost. Directly deploying these models into applications with real-time requirement is still infeasible. Recently, Hinton etal. have shown that the dark knowledge within a powerful teacher model can significantly help the training of a smaller and faster student network. These knowledge are vastly beneficial to improve the generalization ability of the student model. Inspired by their work, we introduce a new type of knowledge -- cross sample similarities for model compression and acceleration. This knowledge can be naturally derived from deep metric learning model. To transfer them, we bring the "learning to rank" technique into deep metric le

Stats

• citations95
• S2 citations—
• github stars0
• HF likes0
• heat score14.87
• arxiv keychen2017darkrank

Related papers

• Deep Metric Learning Using Similarities From Nonlinear Rank Approximations (2019)2.26
• Sharing Matters For Generalization In Deep Metric Learning (2020)8.35
• Deep Metric Learning For Computer Vision: A Brief Overview (2023)6.77
• Data-efficient Ranking Distillation For Image Retrieval (2020)0.00
• Deep Metric Learning Assisted By Intra-variance In A Semi-supervised View Of Learning (2023)5.24
• Dynamic Sampling For Deep Metric Learning (2020)5.84
• Asymmetric Metric Learning For Knowledge Transfer (2020)10.48
• Learning Intra-batch Connections For Deep Metric Learning (2021)0.00