Towards Interpretable Deep Metric Learning With Structural Matching

Abstract

How do the neural networks distinguish two images? It is of critical importance to understand the matching mechanism of deep models for developing reliable intelligent systems for many risky visual applications such as surveillance and access control. However, most existing deep metric learning methods match the images by comparing feature vectors, which ignores the spatial structure of images and thus lacks interpretability. In this paper, we present a deep interpretable metric learning (DIML) method for more transparent embedding learning. Unlike conventional metric learning methods based on feature vector comparison, we propose a structural matching strategy that explicitly aligns the spatial embeddings by computing an optimal matching flow between feature maps of the two images. Our method enables deep models to learn metrics in a more human-friendly way, where the similarity of two images can be decomposed to several part-wise similarities and their contributions to the overall si

Stats

• citations37
• S2 citations—
• github stars101
• HF likes0
• heat score15.87
• arxiv keyzhao2021towards

Related papers

• Visual Explanation For Deep Metric Learning (2019)14.36
• Hierarchical Metric Learning And Matching For 2D And 3D Geometric Correspondences (2018)11.39
• Deep Metric Learning For Computer Vision: A Brief Overview (2023)6.77
• Integrating Language Guidance Into Vision-based Deep Metric Learning (2022)14.04
• Sharing Matters For Generalization In Deep Metric Learning (2020)8.35
• Dynamic Sampling For Deep Metric Learning (2020)5.84
• Introspective Deep Metric Learning (2023)13.85
• Deep Metric Learning Using Similarities From Nonlinear Rank Approximations (2019)2.26