Modalink: Unifying Modalities For Efficient Image-to-pointcloud Place Recognition

Abstract

Place recognition is an important task for robots and autonomous cars to localize themselves and close loops in pre-built maps. While single-modal sensor-based methods have shown satisfactory performance, cross-modal place recognition that retrieving images from a point-cloud database remains a challenging problem. Current cross-modal methods transform images into 3D points using depth estimation for modality conversion, which are usually computationally intensive and need expensive labeled data for depth supervision. In this work, we introduce a fast and lightweight framework to encode images and point clouds into place-distinctive descriptors. We propose an effective Field of View (FoV) transformation module to convert point clouds into an analogous modality as images. This module eliminates the necessity for depth estimation and helps subsequent modules achieve real-time performance. We further design a non-negative factorization-based encoder to extract mutually consistent semantic

Stats

• citations6
• S2 citations—
• github stars21
• HF likes0
• heat score9.02
• arxiv keyxie2024modalink

Related papers

• Uniloc: Towards Universal Place Recognition Using Any Single Modality (2024)0.00
• VXP: Voxel-cross-pixel Large-scale Image-lidar Place Recognition (2024)5.24
• Crossloc3d: Aerial-ground Cross-source 3D Place Recognition (2023)9.23
• Range And Bird's Eye View Fused Cross-modal Visual Place Recognition (2025)0.00
• Graph-based Non-linear Least Squares Optimization For Visual Place Recognition In Changing Environments (2020)7.16
• Pointnetvlad: Deep Point Cloud Based Retrieval For Large-scale Place Recognition (2018)25.45
• Fast, Compact And Highly Scalable Visual Place Recognition Through Sequence-based Matching Of Overloaded Representations (2020)9.41
• Modality-aware Feature Matching: A Comprehensive Review Of Single- And Cross-modality Techniques (2025)0.00