Cross-modal Deep Metric Learning With Multi-task Regularization

Abstract

DNN-based cross-modal retrieval has become a research hotspot, by which users can search results across various modalities like image and text. However, existing methods mainly focus on the pairwise correlation and reconstruction error of labeled data. They ignore the semantically similar and dissimilar constraints between different modalities, and cannot take advantage of unlabeled data. This paper proposes Cross-modal Deep Metric Learning with Multi-task Regularization (CDMLMR), which integrates quadruplet ranking loss and semi-supervised contrastive loss for modeling cross-modal semantic similarity in a unified multi-task learning architecture. The quadruplet ranking loss can model the semantically similar and dissimilar constraints to preserve cross-modal relative similarity ranking information. The semi-supervised contrastive loss is able to maximize the semantic similarity on both labeled and unlabeled data. Compared to the existing methods, CDMLMR exploits not only the similarit

Stats

• citations15
• S2 citations—
• github stars0
• HF likes0
• heat score9.03
• arxiv keyhuang2017cross

Related papers

• Deep Reversible Consistency Learning For Cross-modal Retrieval (2025)7.81
• Adversarial Cross-modal Retrieval Via Learning And Transferring Single-modal Similarities (2019)8.60
• Semi-supervised Cross-modal Retrieval With Label Prediction (2018)11.29
• Directional Statistics-based Deep Metric Learning For Image Classification And Retrieval (2018)13.05
• Multi-level Distance Regularization For Deep Metric Learning (2021)8.09
• Preserving Semantic Neighborhoods For Robust Cross-modal Retrieval (2020)10.07
• Learning Joint Embedding For Cross-modal Retrieval (2019)5.84
• Ranking-based Deep Cross-modal Hashing (2019)13.34