Colbert-att: Late-interaction Meets Attention For Enhanced Retrieval

Abstract

Vector embeddings from pre-trained language models form a core component in Neural Information Retrieval systems across a multitude of knowledge extraction tasks. The paradigm of late interaction, introduced in ColBERT, demonstrates high accuracy along with runtime efficiency. However, the current formulation fails to take into account the attention weights of query and document terms, which intuitively capture the "importance" of similarities between them, that might lead to a better understanding of relevance between the queries and documents. This work proposes ColBERT-Att, to explicitly integrate attention mechanism into the late interaction framework for enhanced retrieval performance. Empirical evaluation of ColBERT-Att depicts improvements in recall accuracy on MS-MARCO as well as on a wide range of BEIR and LoTTE benchmark datasets.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypatel2026colbert

Related papers

• Colbertv2: Effective And Efficient Retrieval Via Lightweight Late Interaction (2021)17.46
• Colbert: Efficient And Effective Passage Search Via Contextualized Late Interaction Over BERT (2020)0.00
• Jina-colbert-v2: A General-purpose Multilingual Late Interaction Retriever (2024)5.24
• Introducing Neural Bag Of Whole-words With Colberter: Contextualized Late Interactions Using Enhanced Reduction (2022)0.00
• Diagnosable Colbert: Debugging Late-interaction Retrieval Models Using A Learned Latent Space As Reference (2026)0.00
• Pylate: Flexible Training And Retrieval For Late Interaction Models (2025)3.58
• Video-colbert: Contextualized Late Interaction For Text-to-video Retrieval (2025)5.24
• LIR: The First Workshop On Late Interaction And Multi Vector Retrieval @ ECIR 2026 (2025)0.00