Shadow Loss: Memory-linear Deep Metric Learning For Efficient Training

Abstract

Deep metric learning objectives (e.g., triplet loss) require storing and comparing high-dimensional embeddings, making the per-batch loss buffer scale as \(O(S\cdot D)\), where \(S\) is the number of samples in a batch and \(D\) is the feature dimension, thus limiting training on memory-constrained hardware. We propose Shadow Loss, a proxy-free, parameter-free objective that measures similarity via scalar projections onto the anchor direction, reducing the loss-specific buffer from \(O(S\cdot D)\) to \(O(S)\) while preserving the triplet structure. We analyze gradients, provide a Lipschitz continuity bound, and show that Shadow Loss penalizes trivial collapse for stable optimization. Across fine-grained retrieval (CUB-200, CARS196), large-scale product retrieval (Stanford Online Products, In-Shop Clothes), and standard/medical benchmarks (CIFAR-10/100, Tiny-ImageNet, HAM-10K, ODIR-5K), Shadow Loss consistently outperforms recent objectives (Triplet, Soft-Margin Triplet, Angular Triplet

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykhan2023shadow

Related papers

• Constellation Loss: Improving The Efficiency Of Deep Metric Learning Loss Functions For Optimal Embedding (2019)0.00
• The Group Loss For Deep Metric Learning (2019)11.39
• Adaptive Additive Classification-based Loss For Deep Metric Learning (2020)0.00
• Variance & Greediness: A Comparative Study Of Metric-learning Losses (2026)0.00
• Three Things To Know About Deep Metric Learning (2024)0.00
• Hierarchical Proxy-based Loss For Deep Metric Learning (2021)10.85
• Ranked List Loss For Deep Metric Learning (2019)7.50
• Unbiased Evaluation Of Deep Metric Learning Algorithms (2019)0.00