Learning Similarity Conditions Without Explicit Supervision

Abstract

Many real-world tasks require models to compare images along multiple similarity conditions (e.g. similarity in color, category or shape). Existing methods often reason about these complex similarity relationships by learning condition-aware embeddings. While such embeddings aid models in learning different notions of similarity, they also limit their capability to generalize to unseen categories since they require explicit labels at test time. To address this deficiency, we propose an approach that jointly learns representations for the different similarity conditions and their contributions as a latent variable without explicit supervision. Comprehensive experiments across three datasets, Polyvore-Outfits, Maryland-Polyvore and UT-Zappos50k, demonstrate the effectiveness of our approach: our model outperforms the state-of-the-art methods, even those that are strongly supervised with pre-defined similarity conditions, on fill-in-the-blank, outfit compatibility prediction and triplet p

Stats

• citations71
• S2 citations—
• github stars0
• HF likes0
• heat score13.93
• arxiv keytan2019learning

Related papers

• Conditional Similarity Networks (2016)15.06
• A Probabilistic Approach For Learning Embeddings Without Supervision (2019)0.00
• Image Similarity Using An Ensemble Of Context-sensitive Models (2024)4.52
• Learning To Embed Semantic Similarity For Joint Image-text Retrieval (2022)7.50
• Pairwise Supervision Can Provably Elicit A Decision Boundary (2020)0.00
• Effectively Leveraging Attributes For Visual Similarity (2021)3.66
• CLAY: Conditional Visual Similarity Modulation In Vision-language Embedding Space (2026)0.00
• Multimodal Representation Learning Conditioned On Semantic Relations (2025)0.00