Integrating Visual And Semantic Similarity Using Hierarchies For Image Retrieval

Abstract

Most of the research in content-based image retrieval (CBIR) focus on developing robust feature representations that can effectively retrieve instances from a database of images that are visually similar to a query. However, the retrieved images sometimes contain results that are not semantically related to the query. To address this, we propose a method for CBIR that captures both visual and semantic similarity using a visual hierarchy. The hierarchy is constructed by merging classes with overlapping features in the latent space of a deep neural network trained for classification, assuming that overlapping classes share high visual and semantic similarities. Finally, the constructed hierarchy is integrated into the distance calculation metric for similarity search. Experiments on standard datasets: CUB-200-2011 and CIFAR100, and a real-life use case using diatom microscopy images show that our method achieves superior performance compared to the existing methods on image retrieval.

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyvenkataramanan2023integrating

Related papers

• Hierarchy-based Image Embeddings For Semantic Image Retrieval (2018)13.84
• Semantic Hierarchy Preserving Deep Hashing For Large-scale Image Retrieval (2019)5.24
• Learning Visual Hierarchies In Hyperbolic Space For Image Retrieval (2024)0.00
• CBIR Using Features Derived By Deep Learning (2020)12.02
• Adaptive Semantic-visual Tree For Hierarchical Embeddings (2020)8.09
• Deep Hashing Learning For Visual And Semantic Retrieval Of Remote Sensing Images (2019)13.55
• An Efficient Image Retrieval Based On Fusion Of Low-level Visual Features (2018)3.58
• Hierarchical Similarity Learning For Language-based Product Image Retrieval (2021)6.93