Visual Similarity Attention

Abstract

While there has been substantial progress in learning suitable distance metrics, these techniques in general lack transparency and decision reasoning, i.e., explaining why the input set of images is similar or dissimilar. In this work, we solve this key problem by proposing the first method to generate generic visual similarity explanations with gradient-based attention. We demonstrate that our technique is agnostic to the specific similarity model type, e.g., we show applicability to Siamese, triplet, and quadruplet models. Furthermore, we make our proposed similarity attention a principled part of the learning process, resulting in a new paradigm for learning similarity functions. We demonstrate that our learning mechanism results in more generalizable, as well as explainable, similarity models. Finally, we demonstrate the generality of our framework by means of experiments on a variety of tasks, including image retrieval, person re-identification, and low-shot semantic segmentation.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzheng2019visual

Related papers

• Towards Visually Explaining Similarity Models (2020)0.00
• Attributable Visual Similarity Learning (2022)11.68
• Visual Explanation For Deep Metric Learning (2019)14.36
• Learning Non-metric Visual Similarity For Image Retrieval (2017)11.58
• Axiomatic Explanations For Visual Search, Retrieval, And Similarity Learning (2021)0.00
• Contextual Visual Similarity (2016)0.00
• Contextual Similarity Aggregation With Self-attention For Visual Re-ranking (2021)0.00
• Learning With Multi-modal Gradient Attention For Explainable Composed Image Retrieval (2023)0.00