Medimageinsight: An Open-source Embedding Model For General Domain Medical Imaging

Abstract

In this work, we present MedImageInsight, an open-source medical imaging embedding model. MedImageInsight is trained on medical images with associated text and labels across a diverse collection of domains, including X-Ray, CT, MRI, dermoscopy, OCT, fundus photography, ultrasound, histopathology, and mammography. Rigorous evaluations demonstrate MedImageInsight's ability to achieve state-of-the-art (SOTA) or human expert level performance across classification, image-image search, and fine-tuning tasks. Specifically, on public datasets, MedImageInsight achieves SOTA in CT 3D medical image retrieval, as well as SOTA in disease classification and search for chest X-ray, dermatology, and OCT imaging. Furthermore, MedImageInsight achieves human expert performance in bone age estimation (on both public and partner data), as well as AUC above 0.9 in most other domains. When paired with a text decoder, MedImageInsight achieves near SOTA level single image report findings generation with less

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycodella2024medimageinsight

Related papers

• Unsupervised Multimodal Representation Learning Across Medical Images And Reports (2018)0.00
• Medical Image Retrieval Using Pretrained Embeddings (2023)7.81
• Radiomicsretrieval: A Customizable Framework For Medical Image Retrieval Using Radiomics Features (2025)2.29
• BIMCV-R: A Landmark Dataset For 3D CT Text-image Retrieval (2024)8.09
• Cardioembed: Domain-specialized Text Embeddings For Clinical Cardiology (2025)0.00
• M3ret: Unleashing Zero-shot Multimodal Medical Image Retrieval Via Self-supervision (2025)0.00
• Medeir: A Specialized Medical Embedding Model For Enhanced Information Retrieval (2025)0.00
• Multi-task Cross-modal Learning For Chest X-ray Image Retrieval (2026)0.00