Towards Unsupervised Sketch-based Image Retrieval

Abstract

The practical value of existing supervised sketch-based image retrieval (SBIR) algorithms is largely limited by the requirement for intensive data collection and labeling. In this paper, we present the first attempt at unsupervised SBIR to remove the labeling cost (both category annotations and sketch-photo pairings) that is conventionally needed for training. Existing single-domain unsupervised representation learning methods perform poorly in this application, due to the unique cross-domain (sketch and photo) nature of the problem. We therefore introduce a novel framework that simultaneously performs sketch-photo domain alignment and semantic-aware representation learning. Technically this is underpinned by introducing joint distribution optimal transport (JDOT) to align data from different domains, which we extend with trainable cluster prototypes and feature memory banks to further improve scalability and efficacy. Extensive experiments show that our framework achieves excellent pe

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhu2021towards

Related papers

• Cross-modal Subspace Learning For Fine-grained Sketch-based Image Retrieval (2017)13.34
• More Photos Are All You Need: Semi-supervised Learning For Fine-grained Sketch Based Image Retrieval (2021)13.23
• Data-free Sketch-based Image Retrieval (2023)13.17
• Exploiting Unlabelled Photos For Stronger Fine-grained SBIR (2023)10.61
• A Zero-shot Framework For Sketch-based Image Retrieval (2018)16.49
• Relation-aware Meta-learning For Zero-shot Sketch-based Image Retrieval (2024)0.00
• Active Learning For Fine-grained Sketch-based Image Retrieval (2023)0.00
• An Efficient Framework For Zero-shot Sketch-based Image Retrieval (2021)13.65