Isearle: Improving Textual Inversion For Zero-shot Composed Image Retrieval

Abstract

Given a query consisting of a reference image and a relative caption, Composed Image Retrieval (CIR) aims to retrieve target images visually similar to the reference one while incorporating the changes specified in the relative caption. The reliance of supervised methods on labor-intensive manually labeled datasets hinders their broad applicability. In this work, we introduce a new task, Zero-Shot CIR (ZS-CIR), that addresses CIR without the need for a labeled training dataset. We propose an approach named iSEARLE (improved zero-Shot composEd imAge Retrieval with textuaL invErsion) that involves mapping the visual information of the reference image into a pseudo-word token in CLIP token embedding space and combining it with the relative caption. To foster research on ZS-CIR, we present an open-domain benchmarking dataset named CIRCO (Composed Image Retrieval on Common Objects in context), the first CIR dataset where each query is labeled with multiple ground truths and a semantic categ

Stats

• citations9
• S2 citations—
• github stars197
• HF likes0
• heat score12.09
• arxiv keyagnolucci2024isearle

Related papers

• Zero-shot Composed Image Retrieval With Textual Inversion (2023)19.84
• From Mapping To Composing: A Two-stage Framework For Zero-shot Composed Image Retrieval (2025)0.00
• SETR: A Two-stage Semantic-enhanced Framework For Zero-shot Composed Image Retrieval (2025)0.00
• Fine-grained Zero-shot Composed Image Retrieval With Complementary Visual-semantic Integration (2026)1.24
• Image2sentence Based Asymmetrical Zero-shot Composed Image Retrieval (2024)0.00
• WISER: Wider Search, Deeper Thinking, And Adaptive Fusion For Training-free Zero-shot Composed Image Retrieval (2026)2.98
• Hycir: Boosting Zero-shot Composed Image Retrieval With Synthetic Labels (2024)0.00
• Imagine And Seek: Improving Composed Image Retrieval With An Imagined Proxy (2024)5.24