Approximate Cluster-based Sparse Document Retrieval With Segmented Maximum Term Weights

Abstract

This paper revisits cluster-based retrieval that partitions the inverted index into multiple groups and skips the index partially at cluster and document levels during online inference using a learned sparse representation. It proposes an approximate search scheme with two parameters to control the rank-safeness competitiveness of pruning with segmented maximum term weights within each cluster. Cluster-level maximum weight segmentation allows an improvement in the rank score bound estimation and threshold-based pruning to be approximately adaptive to bound estimation tightness, resulting in better relevance and efficiency. The experiments with MS MARCO passage ranking and BEIR datasets demonstrate the usefulness of the proposed scheme with a comparison to the baselines. This paper presents the design of this approximate retrieval scheme with rank-safeness analysis, compares clustering and segmentation options, and reports evaluation results.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyqiao2024approximate

Related papers

• Dynamic Superblock Pruning For Fast Learned Sparse Retrieval (2025)5.84
• Investigating The Scalability Of Approximate Sparse Retrieval Algorithms To Massive Datasets (2025)5.84
• Adapting Learned Sparse Retrieval For Long Documents (2023)5.24
• On The Reproducibility Of Learned Sparse Retrieval Adaptations For Long Documents (2025)0.00
• On Approximate Nearest Neighbour Selection For Multi-stage Dense Retrieval (2021)8.35
• Optimizing Guided Traversal For Fast Learned Sparse Retrieval (2023)9.03
• Enhancing Dense Retrievers' Robustness With Group-level Reweighting (2023)0.00
• Efficient Inverted Indexes For Approximate Retrieval Over Learned Sparse Representations (2024)11.67