From Measurement To Mitigation: Quantifying And Reducing Identity Leakage In Image Representation Encoders With Linear Subspace Removal

Abstract

Frozen visual embeddings (e.g., CLIP, DINOv2/v3, SSCD) power retrieval and integrity systems, yet their use on face-containing data is constrained by unmeasured identity leakage and a lack of deployable mitigations. We take an attacker-aware view and contribute: (i) a benchmark of visual embeddings that reports open-set verification at low false-accept rates, a calibrated diffusion-based template inversion check, and face-context attribution with equal-area perturbations; and (ii) propose a one-shot linear projector that removes an estimated identity subspace while preserving the complementary space needed for utility, which for brevity we denote as the identity sanitization projection ISP. Across CelebA-20 and VGGFace2, we show that these encoders are robust under open-set linear probes, with CLIP exhibiting relatively higher leakage than DINOv2/v3 and SSCD, robust to template inversion, and are context-dominant. In addition, we show that ISP drives linear access to near-chance while

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygeorge2026from

Related papers

• Face Identity Unlearning For Retrieval Via Embedding Dispersion (2025)0.00
• Discriminate-and-rectify Encoders: Learning From Image Transformation Sets (2017)0.00
• Isoclip: Decomposing CLIP Projectors For Efficient Intra-modal Alignment (2026)3.06
• Lost In Embeddings: Information Loss In Vision-language Models (2025)0.00
• Cityguard: Graph-aware Private Descriptors For Bias-resilient Identity Search Across Urban Cameras (2026)0.00
• Blind To Position, Biased In Language: Probing Mid-layer Representational Bias In Vision-language Encoders For Zero-shot Language-grounded Spatial Understanding (2025)0.00
• Maskinversion: Localized Embeddings Via Optimization Of Explainability Maps (2024)0.00
• Three-in-one: Robust Enhanced Universal Transferable Anti-facial Retrieval In Online Social Networks (2024)0.00