Asymmetric Metric Learning For Knowledge Transfer

Abstract

Knowledge transfer from large teacher models to smaller student models has recently been studied for metric learning, focusing on fine-grained classification. In this work, focusing on instance-level image retrieval, we study an asymmetric testing task, where the database is represented by the teacher and queries by the student. Inspired by this task, we introduce asymmetric metric learning, a novel paradigm of using asymmetric representations at training. This acts as a simple combination of knowledge transfer with the original metric learning task. We systematically evaluate different teacher and student models, metric learning and knowledge transfer loss functions on the new asymmetric testing as well as the standard symmetric testing task, where database and queries are represented by the same model. We find that plain regression is surprisingly effective compared to more complex knowledge transfer mechanisms, working best in asymmetric testing. Interestingly, our asymmetric metr

Stats

• citations24
• S2 citations—
• github stars0
• HF likes0
• heat score10.48
• arxiv keybudnik2020asymmetric

Related papers

• Large-to-small Image Resolution Asymmetry In Deep Metric Learning (2022)9.62
• Data-efficient Ranking Distillation For Image Retrieval (2020)0.00
• Darkrank: Accelerating Deep Metric Learning Via Cross Sample Similarities Transfer (2017)14.87
• Self-taught Metric Learning Without Labels (2022)9.41
• Learning Metrics From Teachers: Compact Networks For Image Embedding (2019)18.74
• Symmetrical Bidirectional Knowledge Alignment For Zero-shot Sketch-based Image Retrieval (2023)4.52
• Active Metric Learning And Classification Using Similarity Queries (2022)0.00
• Embeddistill: A Geometric Knowledge Distillation For Information Retrieval (2023)0.00