Optimizing Dense Retrieval Model Training With Hard Negatives

Abstract

Ranking has always been one of the top concerns in information retrieval researches. For decades, the lexical matching signal has dominated the ad-hoc retrieval process, but solely using this signal in retrieval may cause the vocabulary mismatch problem. In recent years, with the development of representation learning techniques, many researchers turn to Dense Retrieval (DR) models for better ranking performance. Although several existing DR models have already obtained promising results, their performance improvement heavily relies on the sampling of training examples. Many effective sampling strategies are not efficient enough for practical usage, and for most of them, there still lacks theoretical analysis in how and why performance improvement happens. To shed light on these research questions, we theoretically investigate different training strategies for DR models and try to explain why hard negative sampling performs better than random sampling. Through the analysis, we also fin

Stats

• citations150
• S2 citations—
• github stars0
• HF likes0
• heat score16.34
• arxiv keyzhan2021optimizing

Related papers

• Learning To Retrieve: How To Train A Dense Retrieval Model Effectively And Efficiently (2020)0.00
• Syneg: Llm-driven Synthetic Hard-negatives For Dense Retrieval (2024)0.00
• Enhancing Retrieval Performance: An Ensemble Approach For Hard Negative Mining (2024)0.00
• Efficiently Teaching An Effective Dense Retriever With Balanced Topic Aware Sampling (2021)17.07
• Approximate Nearest Neighbor Negative Contrastive Learning For Dense Text Retrieval (2020)0.00
• Hard Negatives, Hard Lessons: Revisiting Training Data Quality For Robust Information Retrieval With Llms (2025)2.26
• RRRA: Resampling And Reranking Through A Retriever Adapter (2025)0.00
• From Distillation To Hard Negative Sampling: Making Sparse Neural IR Models More Effective (2022)0.00