Zero-shot Composed Image Retrieval Considering Query-target Relationship Leveraging Masked Image-text Pairs

Abstract

This paper proposes a novel zero-shot composed image retrieval (CIR) method considering the query-target relationship by masked image-text pairs. The objective of CIR is to retrieve the target image using a query image and a query text. Existing methods use a textual inversion network to convert the query image into a pseudo word to compose the image and text and use a pre-trained visual-language model to realize the retrieval. However, they do not consider the query-target relationship to train the textual inversion network to acquire information for retrieval. In this paper, we propose a novel zero-shot CIR method that is trained end-to-end using masked image-text pairs. By exploiting the abundant image-text pairs that are convenient to obtain with a masking strategy for learning the query-target relationship, it is expected that accurate zero-shot CIR using a retrieval-focused textual inversion network can be realized. Experimental results show the effectiveness of the proposed meth

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keyzhang2024zero

Related papers

• Pretrain Like Your Inference: Masked Tuning Improves Zero-shot Composed Image Retrieval (2023)2.86
• From Mapping To Composing: A Two-stage Framework For Zero-shot Composed Image Retrieval (2025)0.00
• Zero-shot Composed Text-image Retrieval (2023)0.00
• Zero-shot Composed Image Retrieval With Textual Inversion (2023)19.84
• Multimodal Reasoning Agent For Zero-shot Composed Image Retrieval (2025)0.00
• Modality And Task Adaptation For Enhanced Zero-shot Composed Image Retrieval (2024)0.00
• Isearle: Improving Textual Inversion For Zero-shot Composed Image Retrieval (2024)12.09
• Image2sentence Based Asymmetrical Zero-shot Composed Image Retrieval (2024)0.00