Vidvec: Unlocking Video MLLM Embeddings For Video-text Retrieval

Abstract

Recent studies have adapted generative Multimodal Large Language Models (MLLMs) into embedding extractors for vision tasks, typically through fine-tuning to produce universal representations. However, their performance on video remains inferior to Video Foundation Models (VFMs). In this paper, we focus on leveraging MLLMs for video-text embedding and retrieval. We first conduct a systematic layer-wise analysis, showing that intermediate (pre-trained) MLLM layers already encode substantial task-relevant information. Leveraging this insight, we demonstrate that combining intermediate-layer embeddings with a calibrated MLLM head yields strong zero-shot retrieval performance without any training. Building on these findings, we introduce a lightweight text-based alignment strategy which maps dense video captions to short summaries and enables task-related video-text embedding learning without visual supervision. Remarkably, without any fine-tuning beyond text, our method outperforms current

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytzachor2026vidvec

Related papers

• Vill-e: Video LLM Embeddings For Retrieval (2026)0.00
• Verve: Versatile Retrieval For Videos Via Unified Embeddings (2026)0.00
• Memory Enhanced Embedding Learning For Cross-modal Video-text Retrieval (2021)0.00
• Vlm2vec-v2: Advancing Multimodal Embedding For Videos, Images, And Visual Documents (2025)0.00
• Finevit: Progressively Unlocking Fine-grained Perception With Dense Recaptions (2026)0.00
• MATE: Meet At The Embedding -- Connecting Images With Long Texts (2024)5.24
• Mv-adapter: Multimodal Video Transfer Learning For Video Text Retrieval (2023)9.76
• Context-enhanced Video Moment Retrieval With Large Language Models (2024)5.84