Shapematcher: Self-supervised Joint Shape Canonicalization, Segmentation, Retrieval And Deformation

Abstract

In this paper, we present ShapeMatcher, a unified self-supervised learning framework for joint shape canonicalization, segmentation, retrieval and deformation. Given a partially-observed object in an arbitrary pose, we first canonicalize the object by extracting point-wise affine-invariant features, disentangling inherent structure of the object with its pose and size. These learned features are then leveraged to predict semantically consistent part segmentation and corresponding part centers. Next, our lightweight retrieval module aggregates the features within each part as its retrieval token and compare all the tokens with source shapes from a pre-established database to identify the most geometrically similar shape. Finally, we deform the retrieved shape in the deformation module to tightly fit the input object by harnessing part center guided neural cage deformation. The key insight of ShapeMaker is the simultaneous training of the four highly-associated processes: canonicalizatio

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keydi2023shapematcher

Related papers

• Joint Learning Of 3D Shape Retrieval And Deformation (2021)11.08
• Deep Shape Matching (2017)12.61
• KP-RED: Exploiting Semantic Keypoints For Joint 3D Shape Retrieval And Deformation (2024)8.35
• Parts2words: Learning Joint Embedding Of Point Clouds And Texts By Bidirectional Matching Between Parts And Words (2021)9.96
• Skeleton Merger: An Unsupervised Aligned Keypoint Detector (2021)15.68
• End-to-end Learning Of Keypoint Detector And Descriptor For Pose Invariant 3D Matching (2018)12.40
• Mask2cad: 3D Shape Prediction By Learning To Segment And Retrieve (2020)12.87
• Improving The Matching Of Deformable Objects By Learning To Detect Keypoints (2023)7.74