Improving The Robustness Of Dense Retrievers Against Typos Via Multi-positive Contrastive Learning

Abstract

Dense retrieval has become the new paradigm in passage retrieval. Despite its effectiveness on typo-free queries, it is not robust when dealing with queries that contain typos. Current works on improving the typo-robustness of dense retrievers combine (i) data augmentation to obtain the typoed queries during training time with (ii) additional robustifying subtasks that aim to align the original, typo-free queries with their typoed variants. Even though multiple typoed variants are available as positive samples per query, some methods assume a single positive sample and a set of negative ones per anchor and tackle the robustifying subtask with contrastive learning; therefore, making insufficient use of the multiple positives (typoed queries). In contrast, in this work, we argue that all available positives can be used at the same time and employ contrastive learning that supports multiple positives (multi-positive). Experimental results on two datasets show that our proposed approach of

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keysidiropoulos2024improving

Related papers

• Analysing The Robustness Of Dual Encoders For Dense Retrieval Against Misspellings (2022)9.59
• Typos-aware Bottlenecked Pre-training For Robust Dense Retrieval (2023)5.84
• Typo-robust Representation Learning For Dense Retrieval (2023)7.50
• Noise-robust Dense Retrieval Via Contrastive Alignment Post Training (2023)0.00
• Unsupervised Dense Retrieval With Relevance-aware Contrastive Pre-training (2023)10.44
• More Robust Dense Retrieval With Contrastive Dual Learning (2021)11.88
• Bridging The Training-inference Gap For Dense Phrase Retrieval (2022)2.26
• Unsupervised Dense Information Retrieval With Contrastive Learning (2021)0.00