Use What You Have: Video Retrieval Using Representations From Collaborative Experts
2019 · Yang Liu, Samuel Albanie, Arsha Nagrani, et al.
Abstract
The rapid growth of video on the internet has made searching for video content using natural language queries a significant challenge. Human-generated queries for video datasets `in the wild' vary a lot in terms of degree of specificity, with some queries describing specific details such as the names of famous identities, content from speech, or text available on the screen. Our goal is to condense the multi-modal, extremely high dimensional information from videos into a single, compact video representation for the task of video retrieval using free-form text queries, where the degree of specificity is open-ended. For this we exploit existing knowledge in the form of pre-trained semantic embeddings which include 'general' features such as motion, appearance, and scene features from visual content. We also explore the use of more 'specific' cues from ASR and OCR which are intermittently available for videos and find that these signals remain challenging to use effectively for retriev
Authors
(none)
Tags
Stats
Related papers
- Learning Joint Representations Of Videos And Sentences With Web Image Search (2016)12.93
- Tree-augmented Cross-modal Encoding For Complex-query Video Retrieval (2020)15.57
- Dual Encoding For Video Retrieval By Text (2020)16.05
- Bridging Information Asymmetry In Text-video Retrieval: A Data-centric Approach (2024)0.00
- Semantic Video Moments Retrieval At Scale: A New Task And A Baseline (2022)0.00
- Exploiting Local Indexing And Deep Feature Confidence Scores For Fast Image-to-video Search (2018)2.26
- Verve: Versatile Retrieval For Videos Via Unified Embeddings (2026)0.00
- Generative Recall, Dense Reranking: Learning Multi-view Semantic Ids For Efficient Text-to-video Retrieval (2026)0.00