Composed Image Retrieval Using Contrastive Learning And Task-oriented Clip-based Features

Abstract

Given a query composed of a reference image and a relative caption, the Composed Image Retrieval goal is to retrieve images visually similar to the reference one that integrates the modifications expressed by the caption. Given that recent research has demonstrated the efficacy of large-scale vision and language pre-trained (VLP) models in various tasks, we rely on features from the OpenAI CLIP model to tackle the considered task. We initially perform a task-oriented fine-tuning of both CLIP encoders using the element-wise sum of visual and textual features. Then, in the second stage, we train a Combiner network that learns to combine the image-text features integrating the bimodal information and providing combined features used to perform the retrieval. We use contrastive learning in both stages of training. Starting from the bare CLIP features as a baseline, experimental results show that the task-oriented fine-tuning and the carefully crafted Combiner network are highly effective a

Stats

• citations42
• S2 citations—
• github stars195
• HF likes0
• heat score16.84
• arxiv keybaldrati2023composed

Related papers

• Semantic Compositions Enhance Vision-language Contrastive Learning (2024)0.00
• Compositional Image Retrieval Via Instruction-aware Contrastive Learning (2024)0.00
• Advancing Compositional Awareness In CLIP With Efficient Fine-tuning (2025)0.00
• Bi-directional Training For Composed Image Retrieval Via Text Prompt Learning (2023)15.63
• Learning Visual Composition Through Improved Semantic Guidance (2024)0.00
• COLA: A Benchmark For Compositional Text-to-image Retrieval (2023)0.00
• Improving Composed Image Retrieval Via Contrastive Learning With Scaling Positives And Negatives (2024)11.30
• Compositional Image-text Matching And Retrieval By Grounding Entities (2025)0.60