Partially Does It: Towards Scene-level FG-SBIR With Partial Input

Abstract

We scrutinise an important observation plaguing scene-level sketch research -- that a significant portion of scene sketches are "partial". A quick pilot study reveals: (i) a scene sketch does not necessarily contain all objects in the corresponding photo, due to the subjective holistic interpretation of scenes, (ii) there exists significant empty (white) regions as a result of object-level abstraction, and as a result, (iii) existing scene-level fine-grained sketch-based image retrieval methods collapse as scene sketches become more partial. To solve this "partial" problem, we advocate for a simple set-based approach using optimal transport (OT) to model cross-modal region associativity in a partially-aware fashion. Importantly, we improve upon OT to further account for holistic partialness by comparing intra-modal adjacency matrices. Our proposed method is not only robust to partial scene-sketches but also yields state-of-the-art performance on existing datasets.

Stats

• citations28
• S2 citations—
• github stars0
• HF likes0
• heat score10.97
• arxiv keychowdhury2022partially

Related papers

• Back To The Drawing Board: Rethinking Scene-level Sketch-based Image Retrieval (2025)0.00
• Freeview Sketching: View-aware Fine-grained Sketch-based Image Retrieval (2024)6.34
• Deep Reinforced Attention Regression For Partial Sketch Based Image Retrieval (2021)5.24
• Exploiting Unlabelled Photos For Stronger Fine-grained SBIR (2023)10.61
• Sketch Less For More: On-the-fly Fine-grained Sketch Based Image Retrieval (2020)15.28
• Cross-modal Subspace Learning For Fine-grained Sketch-based Image Retrieval (2017)13.34
• FS-COCO: Towards Understanding Of Freehand Sketches Of Common Objects In Context (2022)11.93
• Active Learning For Fine-grained Sketch-based Image Retrieval (2023)0.00