Domain Adaptation In Multi-view Embedding For Cross-modal Video Retrieval

Abstract

Given a gallery of uncaptioned video sequences, this paper considers the task of retrieving videos based on their relevance to an unseen text query. To compensate for the lack of annotations, we rely instead on a related video gallery composed of video-caption pairs, termed the source gallery, albeit with a domain gap between its videos and those in the target gallery. We thus introduce the problem of Unsupervised Domain Adaptation for Cross-modal Video Retrieval, along with a new benchmark on fine-grained actions. We propose a novel iterative domain alignment method by means of pseudo-labelling target videos and cross-domain (i.e. source-target) ranking. Our approach adapts the embedding space to the target gallery, consistently outperforming source-only as well as marginal and conditional alignment methods.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymunro2021domain

Related papers

• Fine-grained Action Retrieval Through Multiple Parts-of-speech Embeddings (2019)15.62
• Dual Encoding For Video Retrieval By Text (2020)16.05
• Generalized Multi-view Embedding For Visual Recognition And Cross-modal Retrieval (2016)14.69
• Hybrid-learning Video Moment Retrieval Across Multi-domain Labels (2024)0.00
• Cross-modal Embeddings For Video And Audio Retrieval (2018)11.08
• Dual-triplet Metric Learning For Unsupervised Domain Adaptation In Video-based Face Recognition (2020)5.84
• Audio-visual Embedding For Cross-modal Musicvideo Retrieval Through Supervised Deep CCA (2019)11.93
• Aligning Multilingual Word Embeddings For Cross-modal Retrieval Task (2019)2.26