Shapey: Measuring Shape Recognition Capacity Using Nearest Neighbor Matching

Abstract

Object recognition in humans depends primarily on shape cues. We have developed a new approach to measuring the shape recognition performance of a vision system based on nearest neighbor view matching within the system's embedding space. Our performance benchmark, ShapeY, allows for precise control of task difficulty, by enforcing that view matching span a specified degree of 3D viewpoint change and/or appearance change. As a first test case we measured the performance of ResNet50 pre-trained on ImageNet. Matching error rates were high. For example, a 27 degree change in object pitch led ResNet50 to match the incorrect object 45% of the time. Appearance changes were also highly disruptive. Examination of false matches indicates that ResNet50's embedding space is severely "tangled". These findings suggest ShapeY can be a useful tool for charting the progress of artificial vision systems towards human-level shape recognition capabilities.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynam2021shapey

Related papers

• Deep Shape Matching (2017)12.61
• Zero In On Shape: A Generic 2D-3D Instance Similarity Metric Learned From Synthetic Data (2021)5.84
• Fine-tuned But Zero-shot 3D Shape Sketch View Similarity And Retrieval (2023)5.24
• Bayesian Inference Of Bijective Non-rigid Shape Correspondence (2016)0.00
• Learning Discriminative 3D Shape Representations By View Discerning Networks (2018)8.60
• Risa-net: Rotation-invariant Structure-aware Network For Fine-grained 3D Shape Retrieval (2020)5.48
• Extending Deepsdf For Automatic 3D Shape Retrieval And Similarity Transform Estimation (2020)0.00
• Shapematcher: Self-supervised Joint Shape Canonicalization, Segmentation, Retrieval And Deformation (2023)6.77