Revisiting Neural Retrieval On Accelerators

Abstract

Retrieval finds a small number of relevant candidates from a large corpus for information retrieval and recommendation applications. A key component of retrieval is to model (user, item) similarity, which is commonly represented as the dot product of two learned embeddings. This formulation permits efficient inference, commonly known as Maximum Inner Product Search (MIPS). Despite its popularity, dot products cannot capture complex user-item interactions, which are multifaceted and likely high rank. We hence examine non-dot-product retrieval settings on accelerators, and propose \textit\{mixture of logits\} (MoL), which models (user, item) similarity as an adaptive composition of elementary similarity functions. This new formulation is expressive, capable of modeling high rank (user, item) interactions, and further generalizes to the long tail. When combined with a hierarchical retrieval strategy, \textit\{h-indexer\}, we are able to scale up MoL to 100M corpus on a single GPU with lat

Stats

• citations17
• S2 citations—
• github stars0
• HF likes0
• heat score9.41
• arxiv keyzhai2023revisiting

Related papers

• Retrieval With Learned Similarities (2024)0.00
• Deep Retrieval: Learning A Retrievable Structure For Large-scale Recommendations (2020)0.00
• Pylate: Flexible Training And Retrieval For Late Interaction Models (2025)3.58
• Optimizing Compound Retrieval Systems (2025)0.00
• Optimizing Recall Or Relevance? A Multi-task Multi-head Approach For Item-to-item Retrieval In Recommendation (2025)0.00
• Hierarchical Structured Neural Network: Efficient Retrieval Scaling For Large Scale Recommendation (2024)0.00
• Indexing Multimodal Language Models For Large-scale Image Retrieval (2026)0.00
• Lightretriever: A Llm-based Text Retrieval Architecture With Extremely Faster Query Inference (2025)0.00