Lipsim: A Provably Robust Perceptual Similarity Metric

Abstract

Recent years have seen growing interest in developing and applying perceptual similarity metrics. Research has shown the superiority of perceptual metrics over pixel-wise metrics in aligning with human perception and serving as a proxy for the human visual system. On the other hand, as perceptual metrics rely on neural networks, there is a growing concern regarding their resilience, given the established vulnerability of neural networks to adversarial attacks. It is indeed logical to infer that perceptual metrics may inherit both the strengths and shortcomings of neural networks. In this work, we demonstrate the vulnerability of state-of-the-art perceptual similarity metrics based on an ensemble of ViT-based feature extractors to adversarial attacks. We then propose a framework to train a robust perceptual similarity metric called LipSim (Lipschitz Similarity Metric) with provable guarantees. By leveraging 1-Lipschitz neural networks as the backbone, LipSim provides guarded areas aroun

Stats

• citations0
• S2 citations—
• github stars23
• HF likes0
• heat score2.76
• arxiv keyghazanfari2023lipsim

Related papers

• Adversarially Robust CLIP Models Can Induce Better (robust) Perceptual Metrics (2025)3.58
• Dreamsim: Learning New Dimensions Of Human Visual Similarity Using Synthetic Data (2023)5.84
• Identifying And Mitigating Flaws Of Deep Perceptual Similarity Metrics (2022)4.18
• Needle In A Haystack, Fast: Benchmarking Image Perceptual Similarity Metrics At Scale (2022)0.00
• Id-sim: An Identity-focused Similarity Metric (2026)0.00
• Procsim: Proxy-based Confidence For Robust Similarity Learning (2023)0.00
• Deep Perceptual Similarity Is Adaptable To Ambiguous Contexts (2023)1.40
• Learning Non-metric Visual Similarity For Image Retrieval (2017)11.58