Zero-shot Dense Retrieval With Momentum Adversarial Domain Invariant Representations

Abstract

Dense retrieval (DR) methods conduct text retrieval by first encoding texts in the embedding space and then matching them by nearest neighbor search. This requires strong locality properties from the representation space, i.e, the close allocations of each small group of relevant texts, which are hard to generalize to domains without sufficient training data. In this paper, we aim to improve the generalization ability of DR models from source training domains with rich supervision signals to target domains without any relevant labels, in the zero-shot setting. To achieve that, we propose Momentum adversarial Domain Invariant Representation learning (MoDIR), which introduces a momentum method in the DR training process to train a domain classifier distinguishing source versus target, and then adversarially updates the DR encoder to learn domain invariant representations. Our experiments show that MoDIR robustly outperforms its baselines on 10+ ranking datasets from the BEIR benchmark in

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keyxin2021zero

Related papers

• Disentangled Modeling Of Domain And Relevance For Adaptable Dense Retrieval (2022)0.00
• Mixture Of Experts Approaches In Dense Retrieval Tasks (2025)0.95
• COCO-DR: Combating Distribution Shifts In Zero-shot Dense Retrieval With Contrastive And Distributionally Robust Learning (2022)12.85
• A Representation Sharpening Framework For Zero Shot Dense Retrieval (2025)0.00
• Interpreting Dense Retrieval As Mixture Of Topics (2021)0.00
• Black-box Adversarial Attacks Against Dense Retrieval Models: A Multi-view Contrastive Learning Method (2023)9.92
• Approximate Nearest Neighbor Negative Contrastive Learning For Dense Text Retrieval (2020)0.00
• Towards Dynamic Dense Retrieval With Routing Strategy (2026)0.00