Disco: LLM Knowledge Distillation For Efficient Sparse Retrieval In Conversational Search

Abstract

Conversational Search (CS) involves retrieving relevant documents from a corpus while considering the conversational context, integrating retrieval with context modeling. Recent advancements in Large Language Models (LLMs) have significantly enhanced CS by enabling query rewriting based on conversational context. However, employing LLMs during inference poses efficiency challenges. Existing solutions mitigate this issue by distilling embeddings derived from human-rewritten queries, focusing primarily on learning the context modeling task. These methods, however, often separate the contrastive retrieval task from the distillation process, treating it as an independent loss term. To overcome these limitations, we introduce DiSCo (Distillation of Sparse Conversational retrieval), a novel approach that unifies retrieval and context modeling through a relaxed distillation objective. Instead of relying exclusively on representation learning, our method distills similarity scores between conv

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keylupart2024disco

Related papers

• Scaling Sparse And Dense Retrieval In Decoder-only Llms (2025)6.34
• Cosplade: Contextualizing SPLADE For Conversational Information Retrieval (2023)6.77
• CSPLADE: Learned Sparse Retrieval With Causal Language Models (2025)0.00
• Few-shot Conversational Dense Retrieval (2021)16.68
• SLQ: Bridging Modalities Via Shared Latent Queries For Retrieval With Frozen Mllms (2026)0.00
• CONVERSER: Few-shot Conversational Dense Retrieval With Synthetic Data Generation (2023)8.25
• Contextualized Query Embeddings For Conversational Search (2021)10.21
• Sparse And Dense Retrievers Learn Better Together: Joint Sparse-dense Optimization For Text-image Retrieval (2025)0.00