Comparing Contrastive And Triplet Loss: Variance Analysis And Optimization Behavior

Abstract

Contrastive loss and triplet loss are widely used objectives in deep metric learning, yet their effects on representation quality remain insufficiently understood. We present a theoretical and empirical comparison of these losses, focusing on intra- and inter-class variance and optimization behavior (e.g., greedy updates). Through task-specific experiments with consistent settings on synthetic data and real datasets-MNIST, CIFAR-10-it is shown that triplet loss preserves greater variance within and across classes, supporting finer-grained distinctions in the learned representations. In contrast, contrastive loss tends to compact intra-class embeddings, which may obscure subtle semantic differences. To better understand their optimization dynamics, By examining loss-decay rate, active ratio, and gradient norm, we find that contrastive loss drives many small updates early on, while triplet loss produces fewer but stronger updates that sustain learning on hard examples. Finally, across bo

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzeng2025comparing

Related papers

• Variance & Greediness: A Comparative Study Of Metric-learning Losses (2026)0.00
• Constellation Loss: Improving The Efficiency Of Deep Metric Learning Loss Functions For Optimal Embedding (2019)0.00
• Unified Loss Of Pair Similarity Optimization For Vision-language Retrieval (2022)0.00
• Triplet-center Loss For Multi-view 3D Object Retrieval (2018)18.70
• Shadow Loss: Memory-linear Deep Metric Learning For Efficient Training (2023)0.00
• Hard Negative Examples Are Hard, But Useful (2020)14.76
• Unbiased Evaluation Of Deep Metric Learning Algorithms (2019)0.00
• A Quadruplet Loss For Enforcing Semantically Coherent Embeddings In Multi-output Classification Problems (2020)6.77