Location Field Descriptors: Single Image 3D Model Retrieval In The Wild

Abstract

We present Location Field Descriptors, a novel approach for single image 3D model retrieval in the wild. In contrast to previous methods that directly map 3D models and RGB images to an embedding space, we establish a common low-level representation in the form of location fields from which we compute pose invariant 3D shape descriptors. Location fields encode correspondences between 2D pixels and 3D surface coordinates and, thus, explicitly capture 3D shape and 3D pose information without appearance variations which are irrelevant for the task. This early fusion of 3D models and RGB images results in three main advantages: First, the bottleneck location field prediction acts as a regularizer during training. Second, major parts of the system benefit from training on a virtually infinite amount of synthetic data. Finally, the predicted location fields are visually interpretable and unblackbox the system. We evaluate our proposed approach on three challenging real-world datasets (Pix3D,

Stats

• citations32
• S2 citations—
• github stars0
• HF likes0
• heat score11.39
• arxiv keygrabner2019location

Related papers

• Multiview Image-based Localization (2025)0.00
• 3D Pose Estimation And 3D Model Retrieval For Objects In The Wild (2018)15.25
• DH3D: Deep Hierarchical 3D Descriptors For Robust Large-scale 6dof Relocalization (2020)14.76
• Sparse-to-dense Hypercolumn Matching For Long-term Visual Localization (2019)12.99
• Crossloc3d: Aerial-ground Cross-source 3D Place Recognition (2023)9.23
• Img2loc: Revisiting Image Geolocalization Using Multi-modality Foundation Models And Image-based Retrieval-augmented Generation (2024)9.23
• Leveraging Local And Global Descriptors In Parallel To Search Correspondences For Visual Localization (2020)8.82
• Latformer: Locality-aware Point-view Fusion Transformer For 3D Shape Recognition (2021)6.34