Zero-shot Whole Slide Image Retrieval In Histopathology Using Embeddings Of Foundation Models

Abstract

We have tested recently published foundation models for histopathology for image retrieval. We report macro average of F1 score for top-1 retrieval, majority of top-3 retrievals, and majority of top-5 retrievals. We perform zero-shot retrievals, i.e., we do not alter embeddings and we do not train any classifier. As test data, we used diagnostic slides of TCGA, The Cancer Genome Atlas, consisting of 23 organs and 117 cancer subtypes. As a search platform we used Yottixel that enabled us to perform WSI search using patches. Achieved F1 scores show low performance, e.g., for top-5 retrievals, 27% +/- 13% (Yottixel-DenseNet), 42% +/- 14% (Yottixel-UNI), 40%+/-13% (Yottixel-Virchow), 41%+/-13% (Yottixel-GigaPath), and 41%+/-14% (GigaPath WSI).

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyalfasly2024zero

Related papers

• Yottixel -- An Image Search Engine For Large Archives Of Histopathology Whole Slide Images (2019)15.51
• Multimodal Whole Slide Foundation Model For Pathology (2024)12.99
• Learning Binary And Sparse Permutation-invariant Representations For Fast And Memory Efficient Whole Slide Image Search (2022)7.81
• Accurate And Scalable Multimodal Pathology Retrieval Via Attentive Vision-language Alignment (2025)2.26
• A Similarity Measure Of Histopathology Images By Deep Embeddings (2021)4.52
• Pathalign: A Vision-language Model For Whole Slide Images In Histopathology (2024)0.00
• Self-supervised Similarity Learning For Digital Pathology (2019)0.00
• Medical Image Retrieval Using Pretrained Embeddings (2023)7.81