I^3 Retriever: Incorporating Implicit Interaction In Pre-trained Language Models For Passage Retrieval

Abstract

Passage retrieval is a fundamental task in many information systems, such as web search and question answering, where both efficiency and effectiveness are critical concerns. In recent years, neural retrievers based on pre-trained language models (PLM), such as dual-encoders, have achieved huge success. Yet, studies have found that the performance of dual-encoders are often limited due to the neglecting of the interaction information between queries and candidate passages. Therefore, various interaction paradigms have been proposed to improve the performance of vanilla dual-encoders. Particularly, recent state-of-the-art methods often introduce late-interaction during the model inference process. However, such late-interaction based methods usually bring extensive computation and storage cost on large corpus. Despite their effectiveness, the concern of efficiency and space footprint is still an important factor that limits the application of interaction-based neural retrieval models. T

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keydong2023i

Related papers

• Pylate: Flexible Training And Retrieval For Late Interaction Models (2025)3.58
• Interactive Text-to-image Retrieval With Large Language Models: A Plug-and-play Approach (2024)10.24
• Aggretriever: A Simple Approach To Aggregate Textual Representations For Robust Dense Passage Retrieval (2022)13.22
• Simlm: Pre-training With Representation Bottleneck For Dense Passage Retrieval (2022)20.27
• Colbertv2: Effective And Efficient Retrieval Via Lightweight Late Interaction (2021)17.46
• Dense Passage Retrieval: Is It Retrieving? (2024)6.34
• Investigating Multi-layer Representations For Dense Passage Retrieval (2025)0.00
• LIDER: An Efficient High-dimensional Learned Index For Large-scale Dense Passage Retrieval (2022)0.00