MHTN: Modal-adversarial Hybrid Transfer Network For Cross-modal Retrieval

Abstract

Cross-modal retrieval has drawn wide interest for retrieval across different modalities of data. However, existing methods based on DNN face the challenge of insufficient cross-modal training data, which limits the training effectiveness and easily leads to overfitting. Transfer learning is for relieving the problem of insufficient training data, but it mainly focuses on knowledge transfer only from large-scale datasets as single-modal source domain to single-modal target domain. Such large-scale single-modal datasets also contain rich modal-independent semantic knowledge that can be shared across different modalities. Besides, large-scale cross-modal datasets are very labor-consuming to collect and label, so it is significant to fully exploit the knowledge in single-modal datasets for boosting cross-modal retrieval. This paper proposes modal-adversarial hybrid transfer network (MHTN), which to the best of our knowledge is the first work to realize knowledge transfer from single-modal

Stats

• citations103
• S2 citations—
• github stars0
• HF likes0
• heat score15.13
• arxiv keyhuang2017mhtn

Related papers

• Adversarial Cross-modal Retrieval Via Learning And Transferring Single-modal Similarities (2019)8.60
• Self-supervised Adversarial Hashing Networks For Cross-modal Retrieval (2018)19.56
• CL2CM: Improving Cross-lingual Cross-modal Retrieval Via Cross-lingual Knowledge Transfer (2023)8.60
• Hashgan:attention-aware Deep Adversarial Hashing For Cross Modal Retrieval (2017)15.34
• Learning Joint Embedding For Cross-modal Retrieval (2019)5.84
• Multi-modal Transformer For Video Retrieval (2020)19.47
• Transitive Hashing Network For Heterogeneous Multimedia Retrieval (2016)8.35
• CMIR-NET : A Deep Learning Based Model For Cross-modal Retrieval In Remote Sensing (2019)13.34