Query By Semantic Sketch

Abstract

Sketch-based query formulation is very common in image and video retrieval as these techniques often complement textual retrieval methods that are based on either manual or machine generated annotations. In this paper, we present a retrieval approach that allows to query visual media collections by sketching concept maps, thereby merging sketch-based retrieval with the search for semantic labels. Users can draw a spatial distribution of different concept labels, such as "sky", "sea" or "person" and then use these sketches to find images or video scenes that exhibit a similar distribution of these concepts. Hence, this approach does not only take the semantic concepts themselves into account, but also their semantic relations as well as their spatial context. The efficient vector representation enables efficient retrieval even in large multimedia collections. We have integrated the semantic sketch query mode into our retrieval engine vitrivr and demonstrated its effectiveness.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrossetto2019query

Related papers

• Sketchql Demonstration: Zero-shot Video Moment Querying With Sketches (2024)2.26
• Livesketch: Query Perturbations For Guided Sketch-based Visual Search (2019)12.47
• Sketch Less For More: On-the-fly Fine-grained Sketch Based Image Retrieval (2020)15.28
• You'll Never Walk Alone: A Sketch And Text Duet For Fine-grained Image Retrieval (2024)9.41
• Sketch And Text Synergy: Fusing Structural Contours And Descriptive Attributes For Fine-grained Image Retrieval (2026)0.00
• Back To The Drawing Board: Rethinking Scene-level Sketch-based Image Retrieval (2025)0.00
• Composite Sketch+text Queries For Retrieving Objects With Elusive Names And Complex Interactions (2025)5.84
• Zero-shot Sketch-based Remote Sensing Image Retrieval Based On Multi-level And Attention-guided Tokenization (2024)9.50