Training And Challenging Models For Text-guided Fashion Image Retrieval
2022 · Eric Dodds, Jack Culpepper, Gaurav Srivastava
Abstract
Retrieving relevant images from a catalog based on a query image together with a modifying caption is a challenging multimodal task that can particularly benefit domains like apparel shopping, where fine details and subtle variations may be best expressed through natural language. We introduce a new evaluation dataset, Challenging Fashion Queries (CFQ), as well as a modeling approach that achieves state-of-the-art performance on the existing Fashion IQ (FIQ) dataset. CFQ complements existing benchmarks by including relative captions with positive and negative labels of caption accuracy and conditional image similarity, where others provided only positive labels with a combined meaning. We demonstrate the importance of multimodal pretraining for the task and show that domain-specific weak supervision based on attribute labels can augment generic large-scale pretraining. While previous modality fusion mechanisms lose the benefits of multimodal pretraining, we introduce a residual attenti
Authors
(none)
Tags
Stats
Related papers
- Fad-vlp: Fashion Vision-and-language Pre-training Towards Unified Retrieval And Captioning (2022)7.81
- Fashion IQ: A New Dataset Towards Retrieving Images By Natural Language Feedback (2019)17.43
- Facap: A Large-scale Fashion Dataset For Fine-grained Composed Image Retrieval (2025)0.00
- Fashionmv: Product-level Composed Image Retrieval With Multi-view Fashion Data (2026)2.98
- Progressive Learning For Image Retrieval With Hybrid-modality Queries (2022)10.97
- Modality-agnostic Attention Fusion For Visual Search With Text Feedback (2020)0.00
- A Strong Baseline For Fashion Retrieval With Person Re-identification Models (2020)8.09
- Conversational Fashion Image Retrieval Via Multiturn Natural Language Feedback (2021)11.85