Dyvo: Dynamic Vocabularies For Learned Sparse Retrieval With Entities

Abstract

Learned Sparse Retrieval (LSR) models use vocabularies from pre-trained transformers, which often split entities into nonsensical fragments. Splitting entities can reduce retrieval accuracy and limits the model's ability to incorporate up-to-date world knowledge not included in the training data. In this work, we enhance the LSR vocabulary with Wikipedia concepts and entities, enabling the model to resolve ambiguities more effectively and stay current with evolving knowledge. Central to our approach is a Dynamic Vocabulary (DyVo) head, which leverages existing entity embeddings and an entity retrieval component that identifies entities relevant to a query or document. We use the DyVo head to generate entity weights, which are then merged with word piece weights to create joint representations for efficient indexing and retrieval using an inverted index. In experiments across three entity-rich document ranking datasets, the resulting DyVo model substantially outperforms state-of-the-art

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keynguyen2024dyvo

Related papers

• Early Stage Sparse Retrieval With Entity Linking (2022)6.77
• Evdclip: Improving Vision-language Retrieval With Entity Visual Descriptions From Large Language Models (2025)0.00
• What Are You Token About? Dense Retrieval As Distributions Over The Vocabulary (2022)8.09
• The Role Of Vocabularies In Learning Sparse Representations For Ranking (2025)0.00
• Improved Learned Sparse Retrieval With Corpus-specific Vocabularies (2024)6.34
• Sparse And Dense Retrievers Learn Better Together: Joint Sparse-dense Optimization For Text-image Retrieval (2025)0.00
• Adapting Learned Sparse Retrieval For Long Documents (2023)5.24
• Deeper -- Deep Entity Resolution (2017)16.53