Structure-aware 3D VR Sketch To 3D Shape Retrieval

Abstract

We study the practical task of fine-grained 3D-VR-sketch-based 3D shape retrieval. This task is of particular interest as 2D sketches were shown to be effective queries for 2D images. However, due to the domain gap, it remains hard to achieve strong performance in 3D shape retrieval from 2D sketches. Recent work demonstrated the advantage of 3D VR sketching on this task. In our work, we focus on the challenge caused by inherent inaccuracies in 3D VR sketches. We observe that retrieval results obtained with a triplet loss with a fixed margin value, commonly used for retrieval tasks, contain many irrelevant shapes and often just one or few with a similar structure to the query. To mitigate this problem, we for the first time draw a connection between adaptive margin values and shape similarities. In particular, we propose to use a triplet loss with an adaptive margin value driven by a "fitting gap", which is the similarity of two shapes under structure-preserving deformations. We also co

Stats

• citations6
• S2 citations—
• github stars19
• HF likes0
• heat score8.94
• arxiv keyluo2022structure

Related papers

• Towards 3D Vr-sketch To 3D Shape Retrieval (2022)7.16
• Fine-grained VR Sketching: Dataset And Insights (2022)7.81
• Fine-tuned But Zero-shot 3D Shape Sketch View Similarity And Retrieval (2023)5.24
• Joint Learning Of 3D Shape Retrieval And Deformation (2021)11.08
• Risa-net: Rotation-invariant Structure-aware Network For Fine-grained 3D Shape Retrieval (2020)5.48
• SHREC'22 Track: Sketch-based 3D Shape Retrieval In The Wild (2022)8.60
• Uncertainty-aware Cross-modal Transfer Network For Sketch-based 3D Shape Retrieval (2023)5.24
• Extending Deepsdf For Automatic 3D Shape Retrieval And Similarity Transform Estimation (2020)0.00