Zero Shot Composed Image Retrieval

Abstract

Composed image retrieval (CIR) allows a user to locate a target image by applying a fine-grained textual edit (e.g., ``turn the dress blue'' or ``remove stripes'') to a reference image. Zero-shot CIR, which embeds the image and the text with separate pretrained vision-language encoders, reaches only 20-25% Recall@10 on the FashionIQ benchmark. We improve this by fine-tuning BLIP-2 with a lightweight Q-Former that fuses visual and textual features into a single embedding, raising Recall@10 to 45.6% (shirt), 40.1% (dress), and 50.4% (top-tee) and increasing the average Recall@50 to 67.6%. We also examine Retrieval-DPO, which fine-tunes CLIP's text encoder with a Direct Preference Optimization loss applied to FAISS-mined hard negatives. Despite extensive tuning of the scaling factor, index, and sampling strategy, Retrieval-DPO attains only 0.02% Recall@10 -- far below zero-shot and prompt-tuned baselines -- because it (i) lacks joint image-text fusion, (ii) uses a margin objective misalig

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score1.57
• arxiv keykakarla2025zero

Related papers

• From Mapping To Composing: A Two-stage Framework For Zero-shot Composed Image Retrieval (2025)0.00
• Zero-shot Composed Image Retrieval With Textual Inversion (2023)19.84
• Zero-shot Composed Text-image Retrieval (2023)0.00
• Isearle: Improving Textual Inversion For Zero-shot Composed Image Retrieval (2024)12.09
• Scaling Prompt Instructed Zero Shot Composed Image Retrieval With Image-only Data (2025)0.00
• SETR: A Two-stage Semantic-enhanced Framework For Zero-shot Composed Image Retrieval (2025)0.00
• Fine-grained Zero-shot Composed Image Retrieval With Complementary Visual-semantic Integration (2026)1.24
• Training-free Zero-shot Composed Image Retrieval With Local Concept Reranking (2023)0.00