Predicting Visual Overlap Of Images Through Interpretable Non-metric Box Embeddings

Abstract

To what extent are two images picturing the same 3D surfaces? Even when this is a known scene, the answer typically requires an expensive search across scale space, with matching and geometric verification of large sets of local features. This expense is further multiplied when a query image is evaluated against a gallery, e.g. in visual relocalization. While we don't obviate the need for geometric verification, we propose an interpretable image-embedding that cuts the search in scale space to essentially a lookup. Our approach measures the asymmetric relation between two images. The model then learns a scene-specific measure of similarity, from training examples with known 3D visible-surface overlaps. The result is that we can quickly identify, for example, which test image is a close-up version of another, and by what scale factor. Subsequently, local features need only be detected at that scale. We validate our scene-specific model by showing how this embedding yields competitive

Stats

• citations14
• S2 citations—
• github stars0
• HF likes0
• heat score8.82
• arxiv keyrau2020predicting

Related papers

• Breaking The Frame: Visual Place Recognition By Overlap Prediction (2024)7.80
• Self-localization From Images With Small Overlap (2016)8.35
• Supscene: Scene-structured Overlap Supervision For Image Retrieval In Unconstrained Sfm (2026)2.20
• Crossover: 3D Scene Cross-modal Alignment (2025)4.52
• Towards Interpretable Deep Metric Learning With Structural Matching (2021)15.87
• Corrembed: Evaluating Pre-trained Model Image Similarity Efficacy With A Novel Metric (2023)5.24
• Visual Explanation For Deep Metric Learning (2019)14.36
• Dynamic Visual Semantic Sub-embeddings And Fast Re-ranking (2023)0.00