Efficient Passage Retrieval With Hashing For Open-domain Question Answering

Abstract

Most state-of-the-art open-domain question answering systems use a neural retrieval model to encode passages into continuous vectors and extract them from a knowledge source. However, such retrieval models often require large memory to run because of the massive size of their passage index. In this paper, we introduce Binary Passage Retriever (BPR), a memory-efficient neural retrieval model that integrates a learning-to-hash technique into the state-of-the-art Dense Passage Retriever (DPR) to represent the passage index using compact binary codes rather than continuous vectors. BPR is trained with a multi-task objective over two tasks: efficient candidate generation based on binary codes and accurate reranking based on continuous vectors. Compared with DPR, BPR substantially reduces the memory cost from 65GB to 2GB without a loss of accuracy on two standard open-domain question answering benchmarks: Natural Questions and TriviaQA. Our code and trained models are available at https://gi

Stats

• citations31
• S2 citations—
• github stars174
• HF likes0
• heat score15.77
• arxiv keyyamada2021efficient

Related papers

• Improving Dense Passage Retrieval With Multiple Positive Passages (2025)0.00
• Synthetic Target Domain Supervision For Open Retrieval QA (2022)4.52
• Dense Passage Retrieval In Conversational Search (2025)0.00
• Improving Passage Retrieval With Zero-shot Question Generation (2022)12.87
• Tempretriever: Fusion-based Temporal Dense Passage Retrieval For Time-sensitive Questions (2025)0.00
• Pre-training Multi-modal Dense Retrievers For Outside-knowledge Visual Question Answering (2023)7.50
• HASH-RAG: Bridging Deep Hashing With Retriever For Efficient, Fine Retrieval And Augmented Generation (2025)0.00
• Hyperbolic Representation Learning For Fast And Efficient Neural Question Answering (2017)12.61