Domain Adaptation For Dense Retrieval And Conversational Dense Retrieval Through Self-supervision By Meticulous Pseudo-relevance Labeling

Abstract

Recent studies have demonstrated that the ability of dense retrieval models to generalize to target domains with different distributions is limited, which contrasts with the results obtained with interaction-based models. Prior attempts to mitigate this challenge involved leveraging adversarial learning and query generation approaches, but both approaches nevertheless resulted in limited improvements. In this paper, we propose to combine the query-generation approach with a self-supervision approach in which pseudo-relevance labels are automatically generated on the target domain. To accomplish this, a T5-3B model is utilized for pseudo-positive labeling, and meticulous hard negatives are chosen. We also apply this strategy on conversational dense retrieval model for conversational search. A similar pseudo-labeling approach is used, but with the addition of a query-rewriting module to rewrite conversational queries for subsequent labeling. This proposed approach enables a model's domai

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2024domain

Related papers

• Domain Adaptation For Dense Retrieval Through Self-supervision By Pseudo-relevance Labeling (2022)0.00
• GPL: Generative Pseudo Labeling For Unsupervised Domain Adaptation Of Dense Retrieval (2021)17.47
• Disentangled Modeling Of Domain And Relevance For Adaptable Dense Retrieval (2022)0.00
• CONVERSER: Few-shot Conversational Dense Retrieval With Synthetic Data Generation (2023)8.25
• Learning More From Less: Towards Strengthening Weak Supervision For Ad-hoc Retrieval (2019)5.84
• How To Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval (2023)11.39
• Few-shot Conversational Dense Retrieval (2021)16.68
• Unsupervised Dense Retrieval With Relevance-aware Contrastive Pre-training (2023)10.44