Fine-grained Retrieval Prompt Tuning

Abstract

Fine-grained object retrieval aims to learn discriminative representation to retrieve visually similar objects. However, existing top-performing works usually impose pairwise similarities on the semantic embedding spaces or design a localization sub-network to continually fine-tune the entire model in limited data scenarios, thus resulting in convergence to suboptimal solutions. In this paper, we develop Fine-grained Retrieval Prompt Tuning (FRPT), which steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompting and feature adaptation. Specifically, FRPT only needs to learn fewer parameters in the prompt and adaptation instead of fine-tuning the entire model, thus solving the issue of convergence to suboptimal solutions caused by fine-tuning the entire model. Technically, a discriminative perturbation prompt (DPP) is introduced and deemed as a sample prompting process, which amplifies and even exaggerates some discriminative e

Stats

• citations21
• S2 citations—
• github stars0
• HF likes0
• heat score10.07
• arxiv keywang2022fine

Related papers

• Parameter-efficient Prompt Tuning Makes Generalized And Calibrated Neural Text Retrievers (2022)5.84
• Soft Prompt Tuning For Augmenting Dense Retrieval With Large Language Models (2023)9.41
• Retrieval-augmented Dynamic Prompt Tuning For Incomplete Multimodal Learning (2025)8.87
• DGL: Dynamic Global-local Prompt Tuning For Text-video Retrieval (2024)14.35
• Vop: Text-video Co-operative Prompt Tuning For Cross-modal Retrieval (2022)16.41
• Modular Retrieval For Generalization And Interpretation (2023)0.00
• Pros: Prompting-to-simulate Generalized Knowledge For Universal Cross-domain Retrieval (2023)12.56
• Prompt-guided Attention Head Selection For Focus-oriented Image Retrieval (2025)0.00