More Robust Dense Retrieval With Contrastive Dual Learning

Abstract

Dense retrieval conducts text retrieval in the embedding space and has shown many advantages compared to sparse retrieval. Existing dense retrievers optimize representations of queries and documents with contrastive training and map them to the embedding space. The embedding space is optimized by aligning the matched query-document pairs and pushing the negative documents away from the query. However, in such training paradigm, the queries are only optimized to align to the documents and are coarsely positioned, leading to an anisotropic query embedding space. In this paper, we analyze the embedding space distributions and propose an effective training paradigm, Contrastive Dual Learning for Approximate Nearest Neighbor (DANCE) to learn fine-grained query representations for dense retrieval. DANCE incorporates an additional dual training object of query retrieval, inspired by the classic information retrieval training axiom, query likelihood. With contrastive learning, the dual trainin

Stats

• citations17
• S2 citations—
• github stars16
• HF likes0
• heat score11.88
• arxiv keyli2021more

Related papers

• Approximate Nearest Neighbor Negative Contrastive Learning For Dense Text Retrieval (2020)0.00
• Unsupervised Dense Information Retrieval With Contrastive Learning (2021)0.00
• Noise-robust Dense Retrieval Via Contrastive Alignment Post Training (2023)0.00
• Learning To Retrieve: How To Train A Dense Retrieval Model Effectively And Efficiently (2020)0.00
• What Are You Token About? Dense Retrieval As Distributions Over The Vocabulary (2022)8.09
• Unsupervised Dense Retrieval With Conterfactual Contrastive Learning (2024)0.00
• Conventional Contrastive Learning Often Falls Short: Improving Dense Retrieval With Cross-encoder Listwise Distillation And Synthetic Data (2025)0.00
• Enhancing The Ranking Context Of Dense Retrieval Methods Through Reciprocal Nearest Neighbors (2023)4.52