Multimodal Contextualized Support For Enhancing Video Retrieval System

Abstract

arXiv:2412.07584v2 Announce Type: replace Abstract: Current video retrieval systems, especially those used in competitions, primarily focus on querying individual keyframes or images rather than encoding an entire clip or video segment. However, queries often describe an action or event over a series of frames, not a specific image. This results in insufficient information when analyzing a single frame, leading to less accurate query results. Moreover, extracting embeddings solely from images (keyframes) does not provide enough information for models to encode higher-level, more abstract insights inferred from the video. These models tend to only describe the objects present in the frame, lacking a deeper understanding. In this work, we propose a system that integrates the latest methodologies, introducing a novel pipeline that extracts multimodal data, and incorporate information from multiple frames within a video, enabling the model to abstract higher-level information that capture

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynguyenle2026multimodal

Related papers

• Enhanced Multimodal Video Retrieval System: Integrating Query Expansion And Cross-modal Temporal Event Retrieval (2025)0.00
• Clamr: Contextualized Late-interaction For Multimodal Content Retrieval (2025)0.00
• Towards Efficient And Robust Moment Retrieval System: A Unified Framework For Multi-granularity Models And Temporal Reranking (2025)2.26
• Contextiq: A Multimodal Expert-based Video Retrieval System For Contextual Advertising (2024)0.00
• Tree-augmented Cross-modal Encoding For Complex-query Video Retrieval (2020)15.57
• Unified Interactive Multimodal Moment Retrieval Via Cascaded Embedding-reranking And Temporal-aware Score Fusion (2025)0.00
• Embedding-based Retrieval In Multimodal Content Moderation (2025)2.26
• Multi-modal Transformer For Video Retrieval (2020)19.47