Unsupervised Dense Retrieval With Conterfactual Contrastive Learning

Abstract

Efficiently retrieving a concise set of candidates from a large document corpus remains a pivotal challenge in Information Retrieval (IR). Neural retrieval models, particularly dense retrieval models built with transformers and pretrained language models, have been popular due to their superior performance. However, criticisms have also been raised on their lack of explainability and vulnerability to adversarial attacks. In response to these challenges, we propose to improve the robustness of dense retrieval models by enhancing their sensitivity of fine-graned relevance signals. A model achieving sensitivity in this context should exhibit high variances when documents' key passages determining their relevance to queries have been modified, while maintaining low variances for other changes in irrelevant passages. This sensitivity allows a dense retrieval model to produce robust results with respect to attacks that try to promote documents without actually increasing their relevance. It

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2024unsupervised

Related papers

• Unsupervised Dense Information Retrieval With Contrastive Learning (2021)0.00
• Unsupervised Dense Retrieval With Relevance-aware Contrastive Pre-training (2023)10.44
• Conventional Contrastive Learning Often Falls Short: Improving Dense Retrieval With Cross-encoder Listwise Distillation And Synthetic Data (2025)0.00
• More Robust Dense Retrieval With Contrastive Dual Learning (2021)11.88
• Evaluating Contrastive Models For Instance-based Image Retrieval (2021)5.24
• Noise-robust Dense Retrieval Via Contrastive Alignment Post Training (2023)0.00
• Approximate Nearest Neighbor Negative Contrastive Learning For Dense Text Retrieval (2020)0.00
• Black-box Adversarial Attacks Against Dense Retrieval Models: A Multi-view Contrastive Learning Method (2023)9.92