Adapting Learned Sparse Retrieval For Long Documents

Abstract

Learned sparse retrieval (LSR) is a family of neural retrieval methods that transform queries and documents into sparse weight vectors aligned with a vocabulary. While LSR approaches like Splade work well for short passages, it is unclear how well they handle longer documents. We investigate existing aggregation approaches for adapting LSR to longer documents and find that proximal scoring is crucial for LSR to handle long documents. To leverage this property, we proposed two adaptations of the Sequential Dependence Model (SDM) to LSR: ExactSDM and SoftSDM. ExactSDM assumes only exact query term dependence, while SoftSDM uses potential functions that model the dependence of query terms and their expansion terms (i.e., terms identified using a transformer's masked language modeling head). Experiments on the MSMARCO Document and TREC Robust04 datasets demonstrate that both ExactSDM and SoftSDM outperform existing LSR aggregation approaches for different document length constraints. Sur

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keynguyen2023adapting

Related papers

• On The Reproducibility Of Learned Sparse Retrieval Adaptations For Long Documents (2025)0.00
• Mistral-splade: Llms For Better Learned Sparse Retrieval (2024)0.00
• Scaling Sparse And Dense Retrieval In Decoder-only Llms (2025)6.34
• CSPLADE: Learned Sparse Retrieval With Causal Language Models (2025)0.00
• SV-RAG: Lora-contextualizing Adaptation Of Mllms For Long Document Understanding (2024)0.00
• Learning Retrieval Models With Sparse Autoencoders (2026)0.00
• On The Challenges And Opportunities Of Learned Sparse Retrieval For Code (2026)0.00
• Multimodal Learned Sparse Retrieval For Image Suggestion (2024)0.00