Predicting Efficiency/effectiveness Trade-offs For Dense Vs. Sparse Retrieval Strategy Selection

Abstract

Over the last few years, contextualized pre-trained transformer models such as BERT have provided substantial improvements on information retrieval tasks. Recent approaches based on pre-trained transformer models such as BERT, fine-tune dense low-dimensional contextualized representations of queries and documents in embedding space. While these dense retrievers enjoy substantial retrieval effectiveness improvements compared to sparse retrievers, they are computationally intensive, requiring substantial GPU resources, and dense retrievers are known to be more expensive from both time and resource perspectives. In addition, sparse retrievers have been shown to retrieve complementary information with respect to dense retrievers, leading to proposals for hybrid retrievers. These hybrid retrievers leverage low-cost, exact-matching based sparse retrievers along with dense retrievers to bridge the semantic gaps between query and documents. In this work, we address this trade-off between the c

Stats

• citations31
• S2 citations—
• github stars0
• HF likes0
• heat score11.29
• arxiv keyarabzadeh2021predicting

Related papers

• Early Stage Sparse Retrieval With Entity Linking (2022)6.77
• Faster Learned Sparse Retrieval With Guided Traversal (2022)11.29
• Unsupervised Dense Retrieval With Conterfactual Contrastive Learning (2024)0.00
• Twinbert: Distilling Knowledge To Twin-structured BERT Models For Efficient Retrieval (2020)0.00
• Towards Competitive Search Relevance For Inference-free Learned Sparse Retrievers (2024)0.00
• Shallow Cross-encoders For Low-latency Retrieval (2024)2.26
• Pre-training Vs. Fine-tuning: A Reproducibility Study On Dense Retrieval Knowledge Acquisition (2025)0.95
• Dynamic Trade-off Prediction In Multi-stage Retrieval Systems (2016)11.93