On The Challenges And Opportunities Of Learned Sparse Retrieval For Code

Abstract

Retrieval over large codebases is a key component of modern LLM-based software engineering systems. Existing approaches predominantly rely on dense embedding models, while learned sparse retrieval (LSR) remains largely unexplored for code. However, applying sparse retrieval to code is challenging due to subword fragmentation, semantic gaps between natural-language queries and code, diversity of programming languages and sub-tasks, and the length of code documents, which can harm sparsity and latency. We introduce SPLADE-Code, the first large-scale family of learned sparse retrieval models specialized for code retrieval (600M-8B parameters). Despite a lightweight one-stage training pipeline, SPLADE-Code achieves state-of-the-art performance among retrievers under 1B parameters (75.4 on MTEB Code) and competitive results at larger scales (79.0 with 8B). We show that learned expansion tokens are critical to bridge lexical and semantic matching, and provide a latency analysis showing that

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylupart2026on

Related papers

• Mistral-splade: Llms For Better Learned Sparse Retrieval (2024)0.00
• CSPLADE: Learned Sparse Retrieval With Causal Language Models (2025)0.00
• Practical Code RAG At Scale: Task-aware Retrieval Design Choices Under Compute Budgets (2025)0.00
• Learning Retrieval Models With Sparse Autoencoders (2026)0.00
• The Role Of Vocabularies In Learning Sparse Representations For Ranking (2025)0.00
• Spider: Spatially Informed Dense Embedding Retrieval For Software Issue Localization (2025)0.00
• Adapting Learned Sparse Retrieval For Long Documents (2023)5.24
• Towards A Generalist Code Embedding Model Based On Massive Data Synthesis (2025)8.13