Unsupervised Dense Information Retrieval With Contrastive Learning

Abstract

Recently, information retrieval has seen the emergence of dense retrievers, using neural networks, as an alternative to classical sparse methods based on term-frequency. These models have obtained state-of-the-art results on datasets and tasks where large training sets are available. However, they do not transfer well to new applications with no training data, and are outperformed by unsupervised term-frequency methods such as BM25. In this work, we explore the limits of contrastive learning as a way to train unsupervised dense retrievers and show that it leads to strong performance in various retrieval settings. On the BEIR benchmark our unsupervised model outperforms BM25 on 11 out of 15 datasets for the Recall@100. When used as pre-training before fine-tuning, either on a few thousands in-domain examples or on the large MS~MARCO dataset, our contrastive model leads to improvements on the BEIR benchmark. Finally, we evaluate our approach for multi-lingual retrieval, where training da

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyizacard2021unsupervised

Related papers

• Unsupervised Dense Retrieval With Conterfactual Contrastive Learning (2024)0.00
• Unsupervised Dense Retrieval With Relevance-aware Contrastive Pre-training (2023)10.44
• Evaluating Contrastive Models For Instance-based Image Retrieval (2021)5.24
• Unsupervised Dense Retrieval Training With Web Anchors (2023)3.81
• More Robust Dense Retrieval With Contrastive Dual Learning (2021)11.88
• Conventional Contrastive Learning Often Falls Short: Improving Dense Retrieval With Cross-encoder Listwise Distillation And Synthetic Data (2025)0.00
• Boosting Data Utilization For Multilingual Dense Retrieval (2025)0.00
• Approximate Nearest Neighbor Negative Contrastive Learning For Dense Text Retrieval (2020)0.00