Rethinking Deep Contrastive Learning With Embedding Memory

Abstract

Pair-wise loss functions have been extensively studied and shown to continuously improve the performance of deep metric learning (DML). However, they are primarily designed with intuition based on simple toy examples, and experimentally identifying the truly effective design is difficult in complicated, real-world cases. In this paper, we provide a new methodology for systematically studying weighting strategies of various pair-wise loss functions, and rethink pair weighting with an embedding memory. We delve into the weighting mechanisms by decomposing the pair-wise functions, and study positive and negative weights separately using direct weight assignment. This allows us to study various weighting functions deeply and systematically via weight curves, and identify a number of meaningful, comprehensive and insightful facts, which come up with our key observation on memory-based DML: it is critical to mine hard negatives and discard easy negatives which are less informative and redund

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhang2021rethinking

Related papers

• Ensemble Of Loss Functions To Improve Generalizability Of Deep Metric Learning Methods (2021)7.16
• The General Pair-based Weighting Loss For Deep Metric Learning (2019)0.00
• Learning With Memory-based Virtual Classes For Deep Metric Learning (2021)9.92
• Cross-batch Memory For Embedding Learning (2019)22.13
• Anti-collapse Loss For Deep Metric Learning Based On Coding Rate Metric (2024)0.00
• Llave: Large Language And Vision Embedding Models With Hardness-weighted Contrastive Learning (2025)3.58
• Ranked List Loss For Deep Metric Learning (2019)7.50
• Constellation Loss: Improving The Efficiency Of Deep Metric Learning Loss Functions For Optimal Embedding (2019)0.00