Mistral-splade: Llms For Better Learned Sparse Retrieval

Abstract

Learned Sparse Retrievers (LSR) have evolved into an effective retrieval strategy that can bridge the gap between traditional keyword-based sparse retrievers and embedding-based dense retrievers. At its core, learned sparse retrievers try to learn the most important semantic keyword expansions from a query and/or document which can facilitate better retrieval with overlapping keyword expansions. LSR like SPLADE has typically been using encoder only models with MLM (masked language modeling) style objective in conjunction with known ways of retrieval performance improvement such as hard negative mining, distillation, etc. In this work, we propose to use decoder-only model for learning semantic keyword expansion. We posit, decoder only models that have seen much higher magnitudes of data are better equipped to learn keyword expansions needed for improved retrieval. We use Mistral as the backbone to develop our Learned Sparse Retriever similar to SPLADE and train it on a subset of sentenc

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydoshi2024mistral

Related papers

• On The Challenges And Opportunities Of Learned Sparse Retrieval For Code (2026)0.00
• Learning Retrieval Models With Sparse Autoencoders (2026)0.00
• Adapting Learned Sparse Retrieval For Long Documents (2023)5.24
• CSPLADE: Learned Sparse Retrieval With Causal Language Models (2025)0.00
• The Role Of Vocabularies In Learning Sparse Representations For Ranking (2025)0.00
• Sparton: Fast And Memory-efficient Triton Kernel For Learned Sparse Retrieval (2026)0.00
• Scaling Sparse And Dense Retrieval In Decoder-only Llms (2025)6.34
• Multimodal Learned Sparse Retrieval For Image Suggestion (2024)0.00