Weakly-supervised End-to-end CAD Retrieval To Scan Objects

Abstract

CAD model retrieval to real-world scene observations has shown strong promise as a basis for 3D perception of objects and a clean, lightweight mesh-based scene representation; however, current approaches to retrieve CAD models to a query scan rely on expensive manual annotations of 1:1 associations of CAD-scan objects, which typically contain strong lower-level geometric differences. We thus propose a new weakly-supervised approach to retrieve semantically and structurally similar CAD models to a query 3D scanned scene without requiring any CAD-scan associations, and only object detection information as oriented bounding boxes. Our approach leverages a fully-differentiable top-\(k\) retrieval layer, enabling end-to-end training guided by geometric and perceptual similarity of the top retrieved CAD models to the scan queries. We demonstrate that our weakly-supervised approach can outperform fully-supervised retrieval methods on challenging real-world ScanNet scans, and maintain robustne

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybeyer2022weakly

Related papers

• Joint Embedding Of 3D Scan And CAD Objects (2019)11.08
• Self-supervised Graph Neural Network For Mechanical CAD Retrieval (2024)0.00
• Fastcad: Real-time CAD Retrieval And Alignment From Scans And Videos (2024)6.34
• 'cadsketchnet' -- An Annotated Sketch Dataset For 3D CAD Model Retrieval With Deep Neural Networks (2021)11.19
• Patch2cad: Patchwise Embedding Learning For In-the-wild Shape Retrieval From A Single Image (2021)10.85
• ROCA: Robust CAD Model Retrieval And Alignment From A Single Image (2021)12.61
• Mask2cad: 3D Shape Prediction By Learning To Segment And Retrieve (2020)12.87
• Sketchcleannet -- A Deep Learning Approach To The Enhancement And Correction Of Query Sketches For A 3D CAD Model Retrieval System (2022)9.03