Colpali: Efficient Document Retrieval With Vision Language Models

Abstract

Documents are visually rich structures that convey information through text, but also figures, page layouts, tables, or even fonts. Since modern retrieval systems mainly rely on the textual information they extract from document pages to index documents -often through lengthy and brittle processes-, they struggle to exploit key visual cues efficiently. This limits their capabilities in many practical document retrieval applications such as Retrieval Augmented Generation (RAG). To benchmark current systems on visually rich document retrieval, we introduce the Visual Document Retrieval Benchmark ViDoRe, composed of various page-level retrieval tasks spanning multiple domains, languages, and practical settings. The inherent complexity and performance shortcomings of modern systems motivate a new concept; doing document retrieval by directly embedding the images of the document pages. We release ColPali, a Vision Language Model trained to produce high-quality multi-vector embeddings from i

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfaysse2024colpali

Related papers

• Modernvbert: Towards Smaller Visual Document Retrievers (2025)0.00
• SERVAL: Surprisingly Effective Zero-shot Visual Document Retrieval Powered By Large Vision And Language Models (2025)0.00
• Nemotron Colembed V2: Top-performing Late Interaction Embedding Models For Visual Document Retrieval (2026)0.00
• Globaldoc: A Cross-modal Vision-language Framework For Real-world Document Image Retrieval And Classification (2023)3.58
• MIRACL-VISION: A Large, Multilingual, Visual Document Retrieval Benchmark (2025)0.00
• Reproducibility, Replicability, And Insights Into Visual Document Retrieval With Late Interaction (2025)2.26
• Beyond Patch Aggregation: 3-pass Pyramid Indexing For Vision-enhanced Document Retrieval (2025)0.00
• Vdocrag: Retrieval-augmented Generation Over Visually-rich Documents (2025)6.34