Modality-specific Cross-modal Similarity Measurement With Recurrent Attention Network

Abstract

Nowadays, cross-modal retrieval plays an indispensable role to flexibly find information across different modalities of data. Effectively measuring the similarity between different modalities of data is the key of cross-modal retrieval. Different modalities such as image and text have imbalanced and complementary relationships, which contain unequal amount of information when describing the same semantics. For example, images often contain more details that cannot be demonstrated by textual descriptions and vice versa. Existing works based on Deep Neural Network (DNN) mostly construct one common space for different modalities to find the latent alignments between them, which lose their exclusive modality-specific characteristics. Different from the existing works, we propose modality-specific cross-modal similarity measurement (MCSM) approach by constructing independent semantic space for each modality, which adopts end-to-end framework to directly generate modality-specific cross-moda

Stats

• citations145
• S2 citations—
• github stars0
• HF likes0
• heat score16.23
• arxiv keypeng2017modality

Related papers

• Adversarial Cross-modal Retrieval Via Learning And Transferring Single-modal Similarities (2019)8.60
• Do Cross Modal Systems Leverage Semantic Relationships? (2019)7.16
• Cross-media Similarity Metric Learning With Unified Deep Networks (2017)5.84
• Preserving Semantic Neighborhoods For Robust Cross-modal Retrieval (2020)10.07
• Towards Cross-modal Text-molecule Retrieval With Better Modality Alignment (2024)4.52
• Multimodal Representation Alignment For Cross-modal Information Retrieval (2025)0.00
• Discriminative Supervised Subspace Learning For Cross-modal Retrieval (2022)0.00
• Deep Reversible Consistency Learning For Cross-modal Retrieval (2025)7.81