Semi-supervised Cross-modal Retrieval With Label Prediction

Abstract

Due to abundance of data from multiple modalities, cross-modal retrieval tasks with image-text, audio-image, etc. are gaining increasing importance. Of the different approaches proposed, supervised methods usually give significant improvement over their unsupervised counterparts at the additional cost of labeling or annotation of the training data. Semi-supervised methods are recently becoming popular as they provide an elegant framework to balance the conflicting requirement of labeling cost and accuracy. In this work, we propose a novel deep semi-supervised framework which can seamlessly handle both labeled as well as unlabeled data. The network has two important components: (a) the label prediction component predicts the labels for the unlabeled portion of the data and then (b) a common modality-invariant representation is learned for cross-modal retrieval. The two parts of the network are trained sequentially one after the other. Extensive experiments on three standard benchmark da

Stats

• citations31
• S2 citations—
• github stars0
• HF likes0
• heat score11.29
• arxiv keymandal2018semi

Related papers

• Label Prediction Framework For Semi-supervised Cross-modal Retrieval (2019)5.24
• Swamp: Swapped Assignment Of Multi-modal Pairs For Cross-modal Retrieval (2021)0.00
• CMIR-NET : A Deep Learning Based Model For Cross-modal Retrieval In Remote Sensing (2019)13.34
• Cross-modal Deep Metric Learning With Multi-task Regularization (2017)9.03
• Self-supervised Visual Representations For Cross-modal Retrieval (2019)7.50
• Webly Supervised Joint Embedding For Cross-modal Image-text Retrieval (2018)13.17
• Discriminative Supervised Subspace Learning For Cross-modal Retrieval (2022)0.00
• Adversarial Cross-modal Retrieval Via Learning And Transferring Single-modal Similarities (2019)8.60