A Novel Self-supervised Cross-modal Image Retrieval Method In Remote Sensing

Abstract

Due to the availability of multi-modal remote sensing (RS) image archives, one of the most important research topics is the development of cross-modal RS image retrieval (CM-RSIR) methods that search semantically similar images across different modalities. Existing CM-RSIR methods require the availability of a high quality and quantity of annotated training images. The collection of a sufficient number of reliable labeled images is time consuming, complex and costly in operational scenarios, and can significantly affect the final accuracy of CM-RSIR. In this paper, we introduce a novel self-supervised CM-RSIR method that aims to: i) model mutual-information between different modalities in a self-supervised manner; ii) retain the distributions of modal-specific feature spaces similar to each other; and iii) define the most similar images within each modality without requiring any annotated training image. To this end, we propose a novel objective including three loss functions that simu

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keysumbul2022a

Related papers

• Exploring A Fine-grained Multiscale Method For Cross-modal Remote Sensing Image Retrieval (2022)16.73
• CMIR-NET : A Deep Learning Based Model For Cross-modal Retrieval In Remote Sensing (2019)13.34
• Remote Sensing Cross-modal Text-image Retrieval Based On Global And Local Information (2022)19.48
• Unsupervised Contrastive Hashing For Cross-modal Retrieval In Remote Sensing (2022)13.84
• An Unsupervised Cross-modal Hashing Method Robust To Noisy Training Image-text Correspondences In Remote Sensing (2022)7.16
• Deep Unsupervised Contrastive Hashing For Large-scale Cross-modal Text-image Retrieval In Remote Sensing (2022)0.00
• Region Convolutional Features For Multi-label Remote Sensing Image Retrieval (2018)17.37
• Towards A Multimodal Framework For Remote Sensing Image Change Retrieval And Captioning (2024)8.85