End-to-end Learning Of Keypoint Detector And Descriptor For Pose Invariant 3D Matching

Abstract

Finding correspondences between images or 3D scans is at the heart of many computer vision and image retrieval applications and is often enabled by matching local keypoint descriptors. Various learning approaches have been applied in the past to different stages of the matching pipeline, considering detector, descriptor, or metric learning objectives. These objectives were typically addressed separately and most previous work has focused on image data. This paper proposes an end-to-end learning framework for keypoint detection and its representation (descriptor) for 3D depth maps or 3D scans, where the two can be jointly optimized towards task-specific objectives without a need for separate annotations. We employ a Siamese architecture augmented by a sampling layer and a novel score loss function which in turn affects the selection of region proposals. The positive and negative examples are obtained automatically by sampling corresponding region proposals based on their consistency wit

Stats

• citations44
• S2 citations—
• github stars0
• HF likes0
• heat score12.40
• arxiv keygeorgakis2018end

Related papers

• Learning Local Descriptors By Optimizing The Keypoint-correspondence Criterion: Applications To Face Matching, Learning From Unlabeled Videos And 3d-shape Retrieval (2016)11.75
• Improving The Matching Of Deformable Objects By Learning To Detect Keypoints (2023)7.74
• Deeppoint3d: Learning Discriminative Local Descriptors Using Deep Metric Learning On 3D Point Clouds (2019)9.59
• Ur2kid: Unifying Retrieval, Keypoint Detection, And Keypoint Description Without Local Correspondence Supervision (2020)0.00
• Hierarchical Metric Learning And Matching For 2D And 3D Geometric Correspondences (2018)11.39
• DH3D: Deep Hierarchical 3D Descriptors For Robust Large-scale 6dof Relocalization (2020)14.76
• LCD: Learned Cross-domain Descriptors For 2D-3D Matching (2019)13.74
• KP-RED: Exploiting Semantic Keypoints For Joint 3D Shape Retrieval And Deformation (2024)8.35