Boot And Switch: Alternating Distillation For Zero-shot Dense Retrieval

Abstract

Neural 'dense' retrieval models are state of the art for many datasets, however these models often exhibit limited domain transfer ability. Existing approaches to adaptation are unwieldy, such as requiring explicit supervision, complex model architectures, or massive external models. We present \(\texttt\{ABEL\}\), a simple but effective unsupervised method to enhance passage retrieval in zero-shot settings. Our technique follows a straightforward loop: a dense retriever learns from supervision signals provided by a reranker, and subsequently, the reranker is updated based on feedback from the improved retriever. By iterating this loop, the two components mutually enhance one another's performance. Experimental results demonstrate that our unsupervised \(\texttt\{ABEL\}\) model outperforms both leading supervised and unsupervised retrievers on the BEIR benchmark. Meanwhile, it exhibits strong adaptation abilities to tasks and domains that were unseen during training. By either fine-tun

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjiang2023boot

Related papers

• Injecting Domain Adaptation With Learning-to-hash For Effective And Efficient Zero-shot Dense Retrieval (2022)2.80
• Domain Adaptation For Dense Retrieval Through Self-supervision By Pseudo-relevance Labeling (2022)0.00
• Selecting Which Dense Retriever To Use For Zero-shot Search (2023)6.34
• Laprador: Unsupervised Pretrained Dense Retriever For Zero-shot Text Retrieval (2022)8.82
• Back To Basics: A Simple Recipe For Improving Out-of-domain Retrieval In Dense Encoders (2023)0.00
• Unsupervised Dense Retrieval With Relevance-aware Contrastive Pre-training (2023)10.44
• Unsupervised Dense Information Retrieval With Contrastive Learning (2021)0.00
• Towards Dynamic Dense Retrieval With Routing Strategy (2026)0.00