Ur2kid: Unifying Retrieval, Keypoint Detection, And Keypoint Description Without Local Correspondence Supervision

Abstract

In this paper, we explore how three related tasks, namely keypoint detection, description, and image retrieval can be jointly tackled using a single unified framework, which is trained without the need of training data with point to point correspondences. By leveraging diverse information from sequential layers of a standard ResNet-based architecture, we are able to extract keypoints and descriptors that encode local information using generic techniques such as local activation norms, channel grouping and dropping, and self-distillation. Subsequently, global information for image retrieval is encoded in an end-to-end pipeline, based on pooling of the aforementioned local responses. In contrast to previous methods in local matching, our method does not depend on pointwise/pixelwise correspondences, and requires no such supervision at all i.e. no depth-maps from an SfM model nor manually created synthetic affine transformations. We illustrate that this simple and direct paradigm, is able

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2020ur2kid

Related papers

• Reuse Your Features: Unifying Retrieval And Feature-metric Alignment (2022)1.69
• End-to-end Learning Of Keypoint Detector And Descriptor For Pose Invariant 3D Matching (2018)12.40
• Learning Local Descriptors By Optimizing The Keypoint-correspondence Criterion: Applications To Face Matching, Learning From Unlabeled Videos And 3d-shape Retrieval (2016)11.75
• Unifying Deep Local And Global Features For Image Search (2020)28.10
• Leveraging Local And Global Descriptors In Parallel To Search Correspondences For Visual Localization (2020)8.82
• Enhancing Deformable Local Features By Jointly Learning To Detect And Describe Keypoints (2023)9.59
• Improving The Matching Of Deformable Objects By Learning To Detect Keypoints (2023)7.74
• Deep Image Retrieval: Learning Global Representations For Image Search (2016)19.67