Digging Into Self-supervised Learning Of Feature Descriptors

Abstract

Fully-supervised CNN-based approaches for learning local image descriptors have shown remarkable results in a wide range of geometric tasks. However, most of them require per-pixel ground-truth keypoint correspondence data which is difficult to acquire at scale. To address this challenge, recent weakly- and self-supervised methods can learn feature descriptors from relative camera poses or using only synthetic rigid transformations such as homographies. In this work, we focus on understanding the limitations of existing self-supervised approaches and propose a set of improvements that combined lead to powerful feature descriptors. We show that increasing the search space from in-pair to in-batch for hard negative mining brings consistent improvement. To enhance the discriminativeness of feature descriptors, we propose a coarse-to-fine method for mining local hard negatives from a wider search space by using global visual image descriptors. We demonstrate that a combination of synthetic

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keymelekhov2021digging

Related papers

• Learning Local Descriptors By Optimizing The Keypoint-correspondence Criterion: Applications To Face Matching, Learning From Unlabeled Videos And 3d-shape Retrieval (2016)11.75
• Working Hard To Know Your Neighbor's Margins: Local Descriptor Learning Loss (2017)0.00
• Fine-tuning CNN Image Retrieval With No Human Annotation (2017)22.75
• Learning Geodesic-aware Local Features From RGB-D Images (2022)6.34
• Self-supervising Fine-grained Region Similarities For Large-scale Image Localization (2020)15.85
• Learning-based Dimensionality Reduction For Computing Compact And Effective Local Feature Descriptors (2022)9.99
• CNN Image Retrieval Learns From Bow: Unsupervised Fine-tuning With Hard Examples (2016)18.59
• Adversarial Learning Of Hard Positives For Place Recognition (2022)2.26