Recurrent Binary Embedding For Gpu-enabled Exhaustive Retrieval From Billion-scale Semantic Vectors

Abstract

Rapid advances in GPU hardware and multiple areas of Deep Learning open up a new opportunity for billion-scale information retrieval with exhaustive search. Building on top of the powerful concept of semantic learning, this paper proposes a Recurrent Binary Embedding (RBE) model that learns compact representations for real-time retrieval. The model has the unique ability to refine a base binary vector by progressively adding binary residual vectors to meet the desired accuracy. The refined vector enables efficient implementation of exhaustive similarity computation with bit-wise operations, followed by a near- lossless k-NN selection algorithm, also proposed in this paper. The proposed algorithms are integrated into an end-to-end multi-GPU system that retrieves thousands of top items from over a billion candidates in real-time. The RBE model and the retrieval system were evaluated with data from a major paid search engine. When measured against the state-of-the-art model for binary rep

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keyshan2018recurrent

Related papers

• End-to-end Binary Representation Learning Via Direct Binary Embedding (2017)5.84
• Vectorsearch: Enhancing Document Retrieval With Semantic Embeddings And Optimized Search (2024)0.00
• Progressively Optimized Bi-granular Document Representation For Scalable Embedding Based Retrieval (2022)11.06
• An Efficient Embedding Based Ad Retrieval With Gpu-powered Feature Interaction (2025)0.00
• Search Efficient Binary Network Embedding (2019)3.58
• Generative Recall, Dense Reranking: Learning Multi-view Semantic Ids For Efficient Text-to-video Retrieval (2026)0.00
• Event-enhanced Retrieval In Real-time Search (2024)0.95
• Reinpool: Reinforcement Learning Pooling Multi-vector Embeddings For Retrieval System (2026)0.00