Patch2cad: Patchwise Embedding Learning For In-the-wild Shape Retrieval From A Single Image
2021 · Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, et al.
Abstract
3D perception of object shapes from RGB image input is fundamental towards semantic scene understanding, grounding image-based perception in our spatially 3-dimensional real-world environments. To achieve a mapping between image views of objects and 3D shapes, we leverage CAD model priors from existing large-scale databases, and propose a novel approach towards constructing a joint embedding space between 2D images and 3D CAD models in a patch-wise fashion -- establishing correspondences between patches of an image view of an object and patches of CAD geometry. This enables part similarity reasoning for retrieving similar CADs to a new image view without exact matches in the database. Our patch embedding provides more robust CAD retrieval for shape estimation in our end-to-end estimation of CAD model shape and pose for detected objects in a single input image. Experiments on in-the-wild, complex imagery from ScanNet show that our approach is more robust than state of the art in real-wo
Authors
(none)
Tags
Stats
Related papers
- Mask2cad: 3D Shape Prediction By Learning To Segment And Retrieve (2020)12.87
- Joint Embedding Of 3D Scan And CAD Objects (2019)11.08
- Weakly-supervised End-to-end CAD Retrieval To Scan Objects (2022)0.00
- ROCA: Robust CAD Model Retrieval And Alignment From A Single Image (2021)12.61
- KP-RED: Exploiting Semantic Keypoints For Joint 3D Shape Retrieval And Deformation (2024)8.35
- Joint Learning Of 3D Shape Retrieval And Deformation (2021)11.08
- Fastcad: Real-time CAD Retrieval And Alignment From Scans And Videos (2024)6.34
- Patch-wise Retrieval: A Bag Of Practical Techniques For Instance-level Matching (2025)0.00