Doodle To Search: Practical Zero-shot Sketch-based Image Retrieval

Abstract

In this paper, we investigate the problem of zero-shot sketch-based image retrieval (ZS-SBIR), where human sketches are used as queries to conduct retrieval of photos from unseen categories. We importantly advance prior arts by proposing a novel ZS-SBIR scenario that represents a firm step forward in its practical application. The new setting uniquely recognizes two important yet often neglected challenges of practical ZS-SBIR, (i) the large domain gap between amateur sketch and photo, and (ii) the necessity for moving towards large-scale retrieval. We first contribute to the community a novel ZS-SBIR dataset, QuickDraw-Extended, that consists of 330,000 sketches and 204,000 photos spanning across 110 categories. Highly abstract amateur human sketches are purposefully sourced to maximize the domain gap, instead of ones included in existing datasets that can often be semi-photorealistic. We then formulate a ZS-SBIR framework to jointly model sketches and photos into a common embedding s

Stats

• citations170
• S2 citations—
• github stars0
• HF likes0
• heat score16.75
• arxiv keydey2019doodle

Related papers

• An Efficient Framework For Zero-shot Sketch-based Image Retrieval (2021)13.65
• A Zero-shot Framework For Sketch-based Image Retrieval (2018)16.49
• Domain-smoothing Network For Zero-shot Sketch-based Image Retrieval (2021)13.92
• CLIP For All Things Zero-shot Sketch-based Image Retrieval, Fine-grained Or Not (2023)15.54
• Semantic Adversarial Network For Zero-shot Sketch-based Image Retrieval (2019)10.74
• Sketch Less For More: On-the-fly Fine-grained Sketch Based Image Retrieval (2020)15.28
• Zero-shot Everything Sketch-based Image Retrieval, And In Explainable Style (2023)16.67
• Relation-aware Meta-learning For Zero-shot Sketch-based Image Retrieval (2024)0.00