Towards Dynamic Dense Retrieval With Routing Strategy

Abstract

The \textit\{de facto\} paradigm for applying dense retrieval (DR) to new tasks involves fine-tuning a pre-trained model for a specific task. However, this paradigm has two significant limitations: (1) It is difficult adapt the DR to a new domain if the training dataset is limited. (2) Old DR models are simply replaced by newer models that are trained from scratch when the former are no longer up to date. Especially for scenarios where the model needs to be updated frequently, this paradigm is prohibitively expensive. To address these challenges, we propose a novel dense retrieval approach, termed \textit\{dynamic dense retrieval\} (DDR). DDR uses \textit\{prefix tuning\} as a \textit\{module\} specialized for a specific domain. These modules can then be compositional combined with a dynamic routing strategy, enabling highly flexible domain adaptation in the retrieval part. Extensive evaluation on six zero-shot downstream tasks demonstrates that this approach can surpass DR while uti

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysu2026towards

Related papers

• Disentangled Modeling Of Domain And Relevance For Adaptable Dense Retrieval (2022)0.00
• Are We There Yet? A Decision Framework For Replacing Term Based Retrieval With Dense Retrieval Systems (2022)0.00
• How To Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval (2023)11.39
• Learning To Retrieve: How To Train A Dense Retrieval Model Effectively And Efficiently (2020)0.00
• Dense Passage Retrieval: Is It Retrieving? (2024)6.34
• Interpreting Dense Retrieval As Mixture Of Topics (2021)0.00
• Curriculum Learning For Dense Retrieval Distillation (2022)11.49
• Few-shot Conversational Dense Retrieval (2021)16.68