Dreamsim: Learning New Dimensions Of Human Visual Similarity Using Synthetic Data

Abstract

Current perceptual similarity metrics operate at the level of pixels and patches. These metrics compare images in terms of their low-level colors and textures, but fail to capture mid-level similarities and differences in image layout, object pose, and semantic content. In this paper, we develop a perceptual metric that assesses images holistically. Our first step is to collect a new dataset of human similarity judgments over image pairs that are alike in diverse ways. Critical to this dataset is that judgments are nearly automatic and shared by all observers. To achieve this we use recent text-to-image models to create synthetic pairs that are perturbed along various dimensions. We observe that popular perceptual metrics fall short of explaining our new data, and we introduce a new metric, DreamSim, tuned to better align with human perception. We analyze how our metric is affected by different visual attributes, and find that it focuses heavily on foreground objects and semantic conte

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyfu2023dreamsim

Related papers

• Id-sim: An Identity-focused Similarity Metric (2026)0.00
• Efficient Discovery And Effective Evaluation Of Visual Perceptual Similarity: A Benchmark And Beyond (2023)4.52
• Lipsim: A Provably Robust Perceptual Similarity Metric (2023)2.76
• Needle In A Haystack, Fast: Benchmarking Image Perceptual Similarity Metrics At Scale (2022)0.00
• CSIM: A Copula-based Similarity Index Sensitive To Local Changes For Image Quality Assessment (2024)2.16
• Identifying And Mitigating Flaws Of Deep Perceptual Similarity Metrics (2022)4.18
• Learning Non-metric Visual Similarity For Image Retrieval (2017)11.58
• Visual Similarity Attention (2019)0.00