Using Text To Teach Image Retrieval
2020 · Haoyu Dong, Ze Wang, Qiang Qiu, et al.
Abstract
Image retrieval relies heavily on the quality of the data modeling and the distance measurement in the feature space. Building on the concept of image manifold, we first propose to represent the feature space of images, learned via neural networks, as a graph. Neighborhoods in the feature space are now defined by the geodesic distance between images, represented as graph vertices or manifold samples. When limited images are available, this manifold is sparsely sampled, making the geodesic computation and the corresponding retrieval harder. To address this, we augment the manifold samples with geometrically aligned text, thereby using a plethora of sentences to teach us about images. In addition to extensive results on standard datasets illustrating the power of text to help in image retrieval, a new public dataset based on CLEVR is introduced to quantify the semantic similarity between visual data and text data. The experimental results show that the joint embedding manifold is a robus
Authors
(none)
Tags
Stats
Related papers
- Embedding Arithmetic Of Multimodal Queries For Image Retrieval (2021)9.03
- Webly Supervised Joint Embedding For Cross-modal Image-text Retrieval (2018)13.17
- Composing Text And Image For Image Retrieval - An Empirical Odyssey (2018)18.71
- Revising Image-text Retrieval Via Multi-modal Entailment (2022)0.00
- Improving Image Recognition By Retrieving From Web-scale Image-text Data (2023)9.41
- Image-text Retrieval Via Preserving Main Semantics Of Vision (2023)10.22
- Fine-grained Image Classification And Retrieval By Combining Visual And Locally Pooled Textual Features (2020)10.48
- Scene Graph Based Image Retrieval -- A Case Study On The CLEVR Dataset (2019)0.00