Adversarial Cross-modal Retrieval Via Learning And Transferring Single-modal Similarities

Abstract

Cross-modal retrieval aims to retrieve relevant data across different modalities (e.g., texts vs. images). The common strategy is to apply element-wise constraints between manually labeled pair-wise items to guide the generators to learn the semantic relationships between the modalities, so that the similar items can be projected close to each other in the common representation subspace. However, such constraints often fail to preserve the semantic structure between unpaired but semantically similar items (e.g. the unpaired items with the same class label are more similar than items with different labels). To address the above problem, we propose a novel cross-modal similarity transferring (CMST) method to learn and preserve the semantic relationships between unpaired items in an unsupervised way. The key idea is to learn the quantitative similarities in single-modal representation subspace, and then transfer them to the common representation subspace to establish the semantic relation

Stats

• citations13
• S2 citations—
• github stars0
• HF likes0
• heat score8.60
• arxiv keywen2019adversarial

Related papers

• Discriminative Semantic Transitive Consistency For Cross-modal Learning (2021)0.00
• Deep Reversible Consistency Learning For Cross-modal Retrieval (2025)7.81
• Swamp: Swapped Assignment Of Multi-modal Pairs For Cross-modal Retrieval (2021)0.00
• Discriminative Supervised Subspace Learning For Cross-modal Retrieval (2022)0.00
• Preserving Semantic Neighborhoods For Robust Cross-modal Retrieval (2020)10.07
• CL2CM: Improving Cross-lingual Cross-modal Retrieval Via Cross-lingual Knowledge Transfer (2023)8.60
• Multimodal Representation Alignment For Cross-modal Information Retrieval (2025)0.00
• Modality-specific Cross-modal Similarity Measurement With Recurrent Attention Network (2017)16.23