Colbert: Efficient And Effective Passage Search Via Contextualized Late Interaction Over BERT

Abstract

Recent progress in Natural Language Understanding (NLU) is driving fast-paced advances in Information Retrieval (IR), largely owed to fine-tuning deep language models (LMs) for document ranking. While remarkably effective, the ranking models based on these LMs increase computational cost by orders of magnitude over prior approaches, particularly as they must feed each query-document pair through a massive neural network to compute a single relevance score. To tackle this, we present ColBERT, a novel ranking model that adapts deep LMs (in particular, BERT) for efficient retrieval. ColBERT introduces a late interaction architecture that independently encodes the query and the document using BERT and then employs a cheap yet powerful interaction step that models their fine-grained similarity. By delaying and yet retaining this fine-granular interaction, ColBERT can leverage the expressiveness of deep LMs while simultaneously gaining the ability to pre-compute document representations offl

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykhattab2020colbert

Related papers

• Colbertv2: Effective And Efficient Retrieval Via Lightweight Late Interaction (2021)17.46
• Colbert-att: Late-interaction Meets Attention For Enhanced Retrieval (2026)0.00
• Introducing Neural Bag Of Whole-words With Colberter: Contextualized Late Interactions Using Enhanced Reduction (2022)0.00
• Diagnosable Colbert: Debugging Late-interaction Retrieval Models Using A Learned Latent Space As Reference (2026)0.00
• Jina-colbert-v2: A General-purpose Multilingual Late Interaction Retriever (2024)5.24
• Pylate: Flexible Training And Retrieval For Late Interaction Models (2025)3.58
• Video-colbert: Contextualized Late Interaction For Text-to-video Retrieval (2025)5.24
• Turkcolbert: A Benchmark Of Dense And Late-interaction Models For Turkish Information Retrieval (2025)0.00