Intra-modal Constraint Loss For Image-text Retrieval

Abstract

Cross-modal retrieval has drawn much attention in both computer vision and natural language processing domains. With the development of convolutional and recurrent neural networks, the bottleneck of retrieval across image-text modalities is no longer the extraction of image and text features but an efficient loss function learning in embedding space. Many loss functions try to closer pairwise features from heterogeneous modalities. This paper proposes a method for learning joint embedding of images and texts using an intra-modal constraint loss function to reduce the violation of negative pairs from the same homogeneous modality. Experimental results show that our approach outperforms state-of-the-art bi-directional image-text retrieval methods on Flickr30K and Microsoft COCO datasets. Our code is publicly available: https://github.com/CanonChen/IMC.

Stats

• citations7
• S2 citations—
• github stars5
• HF likes0
• heat score8.33
• arxiv keychen2022intra

Code

• CanonChen/IMC—

Related papers

• Cross The Gap: Exposing The Intra-modal Misalignment In CLIP Via Modality Inversion (2025)3.64
• Revisiting Cross Modal Retrieval (2018)0.00
• Deep Multimodal Image-text Embeddings For Automatic Cross-media Retrieval (2020)0.00
• Preserving Semantic Neighborhoods For Robust Cross-modal Retrieval (2020)10.07
• Cross-modal Coherence For Text-to-image Retrieval (2021)6.77
• Cross-modal Image Retrieval With Deep Mutual Information Maximization (2021)9.59
• Image Search Using Multilingual Texts: A Cross-modal Learning Approach Between Image And Text (2019)0.00
• Cross-modal Center Loss (2020)0.00