Working Notes On Late Interaction Dynamics: Analyzing Targeted Behaviors Of Late Interaction Models

Abstract

While Late Interaction models exhibit strong retrieval performance, many of their underlying dynamics remain understudied, potentially hiding performance bottlenecks. In this work, we focus on two topics in Late Interaction retrieval: a length bias that arises when using multi-vector scoring, and the similarity distribution beyond the best scores pooled by the MaxSim operator. We analyze these behaviors for state-of-the-art models on the NanoBEIR benchmark. Results show that while the theoretical length bias of causal Late Interaction models holds in practice, bi-directional models can also suffer from it in extreme cases. We also note that no significant similarity trend lies beyond the top-1 document token, validating that the MaxSim operator efficiently exploits the token-level similarity scores.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyedy2026working

Related papers

• Pylate: Flexible Training And Retrieval For Late Interaction Models (2025)3.58
• Spike Hijacking In Late-interaction Retrieval (2026)0.00
• Colbert-att: Late-interaction Meets Attention For Enhanced Retrieval (2026)0.00
• An Analysis On Matching Mechanisms And Token Pruning For Late-interaction Models (2024)5.24
• Colbertv2: Effective And Efficient Retrieval Via Lightweight Late Interaction (2021)17.46
• SLIM: Sparsified Late Interaction For Multi-vector Retrieval With Inverted Indexes (2023)7.50
• Efficient Document Ranking With Learnable Late Interactions (2024)0.00
• Col-bandit: Zero-shot Query-time Pruning For Late-interaction Retrieval (2026)0.00