Benchmarking Robustness Of Text-image Composed Retrieval

Abstract

Text-image composed retrieval aims to retrieve the target image through the composed query, which is specified in the form of an image plus some text that describes desired modifications to the input image. It has recently attracted attention due to its ability to leverage both information-rich images and concise language to precisely express the requirements for target images. However, the robustness of these approaches against real-world corruptions or further text understanding has never been studied. In this paper, we perform the first robustness study and establish three new diversified benchmarks for systematic analysis of text-image composed retrieval against natural corruptions in both vision and text and further probe textural understanding. For natural corruption analysis, we introduce two new large-scale benchmark datasets, CIRR-C and FashionIQ-C for testing in open domain and fashion domain respectively, both of which apply 15 visual corruptions and 7 textural corruptions.

Stats

• citations0
• S2 citations—
• github stars12
• HF likes0
• heat score2.23
• arxiv keysun2023benchmarking

Related papers

• Benchmark Granularity And Model Robustness For Image-text Retrieval (2024)0.00
• Composing Text And Image For Image Retrieval - An Empirical Odyssey (2018)18.71
• Rethinking Benchmarks For Cross-modal Image-text Retrieval (2023)13.11
• Context-cir: Learning From Concepts In Text For Composed Image Retrieval (2025)4.67
• COLA: A Benchmark For Compositional Text-to-image Retrieval (2023)0.00
• Image Search With Text Feedback By Additive Attention Compositional Learning (2022)0.00
• A Sanity Check On Composed Image Retrieval (2026)0.00
• Zero-shot Composed Text-image Retrieval (2023)0.00