Dual Encoding For Video Retrieval By Text
2020 · Jianfeng Dong, Xirong Li, Chaoxi Xu, et al.
Abstract
This paper attacks the challenging problem of video retrieval by text. In such a retrieval paradigm, an end user searches for unlabeled videos by ad-hoc queries described exclusively in the form of a natural-language sentence, with no visual example provided. Given videos as sequences of frames and queries as sequences of words, an effective sequence-to-sequence cross-modal matching is crucial. To that end, the two modalities need to be first encoded into real-valued vectors and then projected into a common space. In this paper we achieve this by proposing a dual deep encoding network that encodes videos and queries into powerful dense representations of their own. Our novelty is two-fold. First, different from prior art that resorts to a specific single-level encoder, the proposed network performs multi-level encoding that represents the rich content of both modalities in a coarse-to-fine fashion. Second, different from a conventional common space learning algorithm which is either co
Authors
(none)
Tags
Stats
Related papers
- Tree-augmented Cross-modal Encoding For Complex-query Video Retrieval (2020)15.57
- Unifying Latent And Lexicon Representations For Effective Video-text Retrieval (2024)0.00
- Stacked Convolutional Deep Encoding Network For Video-text Retrieval (2020)7.81
- Memory Enhanced Embedding Learning For Cross-modal Video-text Retrieval (2021)0.00
- Modality-balanced Embedding For Video Retrieval (2022)7.16
- Joint Fusion And Encoding: Advancing Multimodal Retrieval From The Ground Up (2025)0.00
- Verve: Versatile Retrieval For Videos Via Unified Embeddings (2026)0.00
- Multi-modal Transformer For Video Retrieval (2020)19.47