Where's Waldo: Diffusion Features For Personalized Segmentation And Retrieval
2024 · Dvir Samuel, Rami Ben-Ari, Matan Levy, et al.
Abstract
Personalized retrieval and segmentation aim to locate specific instances within a dataset based on an input image and a short description of the reference instance. While supervised methods are effective, they require extensive labeled data for training. Recently, self-supervised foundation models have been introduced to these tasks showing comparable results to supervised methods. However, a significant flaw in these models is evident: they struggle to locate a desired instance when other instances within the same class are presented. In this paper, we explore text-to-image diffusion models for these tasks. Specifically, we propose a novel approach called PDM for Personalized Features Diffusion Matching, that leverages intermediate features of pre-trained text-to-image models for personalization tasks without any additional training. PDM demonstrates superior performance on popular retrieval and segmentation benchmarks, outperforming even supervised methods. We also highlight notable
Authors
(none)
Tags
Stats
Related papers
- Image Retrieval Outperforms Diffusion Models On Data Augmentation (2023)0.00
- Deepdiffusion: Unsupervised Learning Of Retrieval-adapted Representations Via Diffusion-based Ranking On Latent Feature Manifold (2021)5.13
- Diffhash: Text-guided Targeted Attack Via Diffusion Models Against Deep Hashing Image Retrieval (2025)0.00
- Text-to-image Diffusion Models Are Great Sketch-photo Matchmakers (2024)9.41
- Adafuse: Adaptive Diffusion-generated Image And Text Fusion For Interactive Text-to-image Retrieval (2026)0.00
- Genetic Algorithms For The Optimization Of Diffusion Parameters In Content-based Image Retrieval (2019)9.23
- Diff-sbsr: Learning Multimodal Feature-enhanced Diffusion Models For Zero-shot Sketch-based 3D Shape Retrieval (2026)0.00
- Efficient Image Retrieval Via Decoupling Diffusion Into Online And Offline Processing (2018)12.25