Freeview Sketching: View-aware Fine-grained Sketch-based Image Retrieval

Abstract

In this paper, we delve into the intricate dynamics of Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) by addressing a critical yet overlooked aspect -- the choice of viewpoint during sketch creation. Unlike photo systems that seamlessly handle diverse views through extensive datasets, sketch systems, with limited data collected from fixed perspectives, face challenges. Our pilot study, employing a pre-trained FG-SBIR model, highlights the system's struggle when query-sketches differ in viewpoint from target instances. Interestingly, a questionnaire however shows users desire autonomy, with a significant percentage favouring view-specific retrieval. To reconcile this, we advocate for a view-aware system, seamlessly accommodating both view-agnostic and view-specific tasks. Overcoming dataset limitations, our first contribution leverages multi-view 2D projections of 3D objects, instilling cross-modal view awareness. The second contribution introduces a customisable cross-modal featur

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keysain2024freeview

Related papers

• Sketch Less For More: On-the-fly Fine-grained Sketch Based Image Retrieval (2020)15.28
• Fine-grained Instance-level Sketch-based Video Retrieval (2020)11.58
• Cross-modal Hierarchical Modelling For Fine-grained Sketch Based Image Retrieval (2020)6.77
• Adaptive Fine-grained Sketch-based Image Retrieval (2022)9.76
• Active Learning For Fine-grained Sketch-based Image Retrieval (2023)0.00
• Cross-modal Subspace Learning For Fine-grained Sketch-based Image Retrieval (2017)13.34
• Back To The Drawing Board: Rethinking Scene-level Sketch-based Image Retrieval (2025)0.00
• Deep Reinforced Attention Regression For Partial Sketch Based Image Retrieval (2021)5.24