Cross-media Similarity Evaluation For Web Image Retrieval In The Wild

Abstract

In order to retrieve unlabeled images by textual queries, cross-media similarity computation is a key ingredient. Although novel methods are continuously introduced, little has been done to evaluate these methods together with large-scale query log analysis. Consequently, how far have these methods brought us in answering real-user queries is unclear. Given baseline methods that compute cross-media similarity using relatively simple text/image matching, how much progress have advanced models made is also unclear. This paper takes a pragmatic approach to answering the two questions. Queries are automatically categorized according to the proposed query visualness measure, and later connected to the evaluation of multiple cross-media similarity models on three test sets. Such a connection reveals that the success of the state-of-the-art is mainly attributed to their good performance on visual-oriented queries, while these queries account for only a small part of real-user queries. To quan

Stats

• citations18
• S2 citations—
• github stars0
• HF likes0
• heat score9.59
• arxiv keydong2017cross

Related papers

• A New Benchmark And Approach For Fine-grained Cross-media Retrieval (2019)17.33
• Needle In A Haystack, Fast: Benchmarking Image Perceptual Similarity Metrics At Scale (2022)0.00
• Rethinking Benchmarks For Cross-modal Image-text Retrieval (2023)13.11
• Contextual Visual Similarity (2016)0.00
• Efficient Discovery And Effective Evaluation Of Visual Perceptual Similarity: A Benchmark And Beyond (2023)4.52
• Webly Supervised Joint Embedding For Cross-modal Image-text Retrieval (2018)13.17
• Twitter100k: A Real-world Dataset For Weakly Supervised Cross-media Retrieval (2017)13.34
• Deep Multimodal Image-text Embeddings For Automatic Cross-media Retrieval (2020)0.00