Evaluating Pre-trained Convolutional Neural Networks And Foundation Models As Feature Extractors For Content-based Medical Image Retrieval

Abstract

Medical image retrieval refers to the task of finding similar images for given query images in a database, with applications such as diagnosis support. While traditional medical image retrieval relied on clinical metadata, content-based medical image retrieval (CBMIR) depends on image features, which can be extracted automatically or semi-automatically. Many approaches have been proposed for CBMIR, and among them, using pre-trained convolutional neural networks (CNNs) is a widely utilized approach. However, considering the recent advances in the development of foundation models for various computer vision tasks, their application for CBMIR can also be investigated. In this study, we used several pre-trained feature extractors from well-known pre-trained CNNs and pre-trained foundation models and investigated the CBMIR performance on eight types of two-dimensional (2D) and three-dimensional (3D) medical images. Furthermore, we investigated the effect of image size on the CBMIR perform

Stats

• citations12
• S2 citations—
• github stars13
• HF likes0
• heat score10.65
• arxiv keymahbod2024evaluating

Related papers

• Leveraging Foundation Models For Content-based Image Retrieval In Radiology (2024)11.11
• Medical Image Retrieval Using Deep Convolutional Neural Network (2017)19.35
• Medical Image Retrieval Using Pretrained Embeddings (2023)7.81
• Generating Binary Tags For Fast Medical Image Retrieval Based On Convolutional Nets And Radon Transform (2016)12.25
• CBIR Using Features Derived By Deep Learning (2020)12.02
• Content-based Image Retrieval For Multi-class Volumetric Radiology Images: A Benchmark Study (2024)5.24
• CBIDR: A Novel Method For Information Retrieval Combining Image And Data By Means Of TOPSIS Applied To Medical Diagnosis (2024)0.00
• Content-based 3D Image Retrieval And A Colbert-inspired Re-ranking For Tumor Flagging And Staging (2025)0.00