Sketching Without Worrying: Noise-tolerant Sketch-based Image Retrieval

Abstract

Sketching enables many exciting applications, notably, image retrieval. The fear-to-sketch problem (i.e., "I can't sketch") has however proven to be fatal for its widespread adoption. This paper tackles this "fear" head on, and for the first time, proposes an auxiliary module for existing retrieval models that predominantly lets the users sketch without having to worry. We first conducted a pilot study that revealed the secret lies in the existence of noisy strokes, but not so much of the "I can't sketch". We consequently design a stroke subset selector that \{detects noisy strokes, leaving only those\} which make a positive contribution towards successful retrieval. Our Reinforcement Learning based formulation quantifies the importance of each stroke present in a given subset, based on the extent to which that stroke contributes to retrieval. When combined with pre-trained retrieval models as a pre-processing module, we achieve a significant gain of 8%-10% over standard baselines and

Stats

• citations49
• S2 citations—
• github stars0
• HF likes0
• heat score12.74
• arxiv keybhunia2022sketching

Related papers

• Sketch Less For More: On-the-fly Fine-grained Sketch Based Image Retrieval (2020)15.28
• Back To The Drawing Board: Rethinking Scene-level Sketch-based Image Retrieval (2025)0.00
• Active Learning For Fine-grained Sketch-based Image Retrieval (2023)0.00
• Sketch And Text Synergy: Fusing Structural Contours And Descriptive Attributes For Fine-grained Image Retrieval (2026)0.00
• Deep Reinforced Attention Regression For Partial Sketch Based Image Retrieval (2021)5.24
• You'll Never Walk Alone: A Sketch And Text Duet For Fine-grained Image Retrieval (2024)9.41
• A Sketch Is Worth A Thousand Words: Image Retrieval With Text And Sketch (2022)10.35
• Doodle To Search: Practical Zero-shot Sketch-based Image Retrieval (2019)16.75