A New Benchmark And Approach For Fine-grained Cross-media Retrieval

Abstract

Cross-media retrieval is to return the results of various media types corresponding to the query of any media type. Existing researches generally focus on coarse-grained cross-media retrieval. When users submit an image of "Slaty-backed Gull" as a query, coarse-grained cross-media retrieval treats it as "Bird", so that users can only get the results of "Bird", which may include other bird species with similar appearance (image and video), descriptions (text) or sounds (audio), such as "Herring Gull". Such coarse-grained cross-media retrieval is not consistent with human lifestyle, where we generally have the fine-grained requirement of returning the exactly relevant results of "Slaty-backed Gull" instead of "Herring Gull". However, few researches focus on fine-grained cross-media retrieval, which is a highly challenging and practical task. Therefore, in this paper, we first construct a new benchmark for fine-grained cross-media retrieval, which consists of 200 fine-grained subcategorie

Stats

• citations68
• S2 citations—
• github stars58
• HF likes0
• heat score17.33
• arxiv keyhe2019a

Related papers

• Cross-media Similarity Evaluation For Web Image Retrieval In The Wild (2017)9.59
• Deep Learning Techniques For Future Intelligent Cross-media Retrieval (2020)0.00
• Towards An All-purpose Content-based Multimedia Information Retrieval System (2019)0.00
• Rethinking Benchmarks For Cross-modal Image-text Retrieval (2023)13.11
• Beyond Global Similarity: Towards Fine-grained, Multi-condition Multimodal Retrieval (2026)2.20
• Scientific And Technological Information Oriented Semantics-adversarial And Media-adversarial Cross-media Retrieval (2022)0.00
• Twitter100k: A Real-world Dataset For Weakly Supervised Cross-media Retrieval (2017)13.34
• Cross-modal Retrieval: A Systematic Review Of Methods And Future Directions (2023)12.81