Injecting Domain Adaptation With Learning-to-hash For Effective And Efficient Zero-shot Dense Retrieval

Abstract

Dense retrieval overcome the lexical gap and has shown great success in ad-hoc information retrieval (IR). Despite their success, dense retrievers are expensive to serve across practical use cases. For use cases requiring to search from millions of documents, the dense index becomes bulky and requires high memory usage for storing the index. More recently, learning-to-hash (LTH) techniques, for e.g., BPR and JPQ, produce binary document vectors, thereby reducing the memory requirement to efficiently store the dense index. LTH techniques are supervised and finetune the retriever using a ranking loss. They outperform their counterparts, i.e., traditional out-of-the-box vector compression techniques such as PCA or PQ. A missing piece from prior work is that existing techniques have been evaluated only in-domain, i.e., on a single dataset such as MS MARCO. In our work, we evaluate LTH and vector compression techniques for improving the downstream zero-shot retrieval accuracy of the TAS-B d

Stats

• citations0
• S2 citations—
• github stars24
• HF likes0
• heat score2.80
• arxiv keythakur2022injecting

Related papers

• Boot And Switch: Alternating Distillation For Zero-shot Dense Retrieval (2023)0.00
• Dense Retrieval Adaptation Using Target Domain Description (2023)7.50
• Learning To Retrieve: How To Train A Dense Retrieval Model Effectively And Efficiently (2020)0.00
• Precise Zero-shot Dense Retrieval Without Relevance Labels (2022)17.27
• A Representation Sharpening Framework For Zero Shot Dense Retrieval (2025)0.00
• Laprador: Unsupervised Pretrained Dense Retriever For Zero-shot Text Retrieval (2022)8.82
• Selecting Which Dense Retriever To Use For Zero-shot Search (2023)6.34
• Hashing In The Zero Shot Framework With Domain Adaptation (2017)10.21