Multimodal Learning For Scalable Representation Of High-dimensional Medical Data

Abstract

Integrating artificial intelligence (AI) with healthcare data is rapidly transforming medical diagnostics and driving progress toward precision medicine. However, effectively leveraging multimodal data, particularly digital pathology whole slide images (WSIs) and genomic sequencing, remains a significant challenge due to the intrinsic heterogeneity of these modalities and the need for scalable and interpretable frameworks. Existing diagnostic models typically operate on unimodal data, overlooking critical cross-modal interactions that can yield richer clinical insights. We introduce MarbliX (Multimodal Association and Retrieval with Binary Latent Indexed matriX), a self-supervised framework that learns to embed WSIs and immunogenomic profiles into compact, scalable binary codes, termed ``monogram.'' By optimizing a triplet contrastive objective across modalities, MarbliX captures high-resolution patient similarity in a unified latent space, enabling efficient retrieval of clinically re

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyalsaafin2024multimodal

Related papers

• HOMIE: Histopathology Omni-modal Embedding For Pathology Composed Retrieval (2025)0.00
• Benchmarking Vision-language Contrastive Methods For Medical Representation Learning (2024)0.00
• Accurate And Scalable Multimodal Pathology Retrieval Via Attentive Vision-language Alignment (2025)2.26
• Cross-modality Sub-image Retrieval Using Contrastive Multimodal Image Representations (2022)6.32
• Pathalign: A Vision-language Model For Whole Slide Images In Histopathology (2024)0.00
• Multimodal Whole Slide Foundation Model For Pathology (2024)12.99
• Multi-task Cross-modal Learning For Chest X-ray Image Retrieval (2026)0.00
• MRIS: A Multi-modal Retrieval Approach For Image Synthesis On Diverse Modalities (2023)3.58