Parameter-efficient Prompt Tuning Makes Generalized And Calibrated Neural Text Retrievers

Abstract

Prompt tuning attempts to update few task-specific parameters in pre-trained models. It has achieved comparable performance to fine-tuning of the full parameter set on both language understanding and generation tasks. In this work, we study the problem of prompt tuning for neural text retrievers. We introduce parameter-efficient prompt tuning for text retrieval across in-domain, cross-domain, and cross-topic settings. Through an extensive analysis, we show that the strategy can mitigate the two issues -- parameter-inefficiency and weak generalizability -- faced by fine-tuning based retrieval methods. Notably, it can significantly improve the out-of-domain zero-shot generalization of the retrieval models. By updating only 0.1% of the model parameters, the prompt tuning strategy can help retrieval models achieve better generalization performance than traditional methods in which all parameters are updated. Finally, to facilitate research on retrievers' cross-topic generalizability, we cu

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keytam2022parameter

Related papers

• Fine-grained Retrieval Prompt Tuning (2022)10.07
• DGL: Dynamic Global-local Prompt Tuning For Text-video Retrieval (2024)14.35
• Pros: Prompting-to-simulate Generalized Knowledge For Universal Cross-domain Retrieval (2023)12.56
• Soft Prompt Tuning For Augmenting Dense Retrieval With Large Language Models (2023)9.41
• Vop: Text-video Co-operative Prompt Tuning For Cross-modal Retrieval (2022)16.41
• Modular Retrieval For Generalization And Interpretation (2023)0.00
• Parameter-efficient Neural Reranking For Cross-lingual And Multilingual Retrieval (2022)0.00
• Refining Joint Text And Source Code Embeddings For Retrieval Task With Parameter-efficient Fine-tuning (2024)0.00