Exploring \(\ell_0\) Sparsification For Inference-free Sparse Retrievers

Abstract

With increasing demands for efficiency, information retrieval has developed a branch of sparse retrieval, further advancing towards inference-free retrieval where the documents are encoded during indexing time and there is no model-inference for queries. Existing sparse retrieval models rely on FLOPS regularization for sparsification, while this mechanism was originally designed for Siamese encoders, it is considered to be suboptimal in inference-free scenarios which is asymmetric. Previous attempts to adapt FLOPS for inference-free scenarios have been limited to rule-based methods, leaving the potential of sparsification approaches for inference-free retrieval models largely unexplored. In this paper, we explore \(\ell_0\) inspired sparsification manner for inference-free retrievers. Through comprehensive out-of-domain evaluation on the BEIR benchmark, our method achieves state-of-the-art performance among inference-free sparse retrieval models and is comparable to leading Siamese spa

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyshen2025exploring

Related papers

• Towards Competitive Search Relevance For Inference-free Learned Sparse Retrievers (2024)0.00
• Efficient Inverted Indexes For Approximate Retrieval Over Learned Sparse Representations (2024)11.67
• From Distillation To Hard Negative Sampling: Making Sparse Neural IR Models More Effective (2022)0.00
• Scaling Sparse And Dense Retrieval In Decoder-only Llms (2025)6.34
• The Role Of Vocabularies In Learning Sparse Representations For Ranking (2025)0.00
• SLIM: Sparsified Late Interaction For Multi-vector Retrieval With Inverted Indexes (2023)7.50
• Improved Learned Sparse Retrieval With Corpus-specific Vocabularies (2024)6.34
• Learning Retrieval Models With Sparse Autoencoders (2026)0.00