Exploiting Unlabelled Photos For Stronger Fine-grained SBIR

Abstract

This paper advances the fine-grained sketch-based image retrieval (FG-SBIR) literature by putting forward a strong baseline that overshoots prior state-of-the-arts by ~11%. This is not via complicated design though, but by addressing two critical issues facing the community (i) the gold standard triplet loss does not enforce holistic latent space geometry, and (ii) there are never enough sketches to train a high accuracy model. For the former, we propose a simple modification to the standard triplet loss, that explicitly enforces separation amongst photos/sketch instances. For the latter, we put forward a novel knowledge distillation module can leverage photo data for model training. Both modules are then plugged into a novel plug-n-playable training paradigm that allows for more stable training. More specifically, for (i) we employ an intra-modal triplet loss amongst sketches to bring sketches of the same instance closer from others, and one more amongst photos to push away different

Stats

• citations25
• S2 citations—
• github stars0
• HF likes0
• heat score10.61
• arxiv keysain2023exploiting

Related papers

• More Photos Are All You Need: Semi-supervised Learning For Fine-grained Sketch Based Image Retrieval (2021)13.23
• Sketch Less For More: On-the-fly Fine-grained Sketch Based Image Retrieval (2020)15.28
• Active Learning For Fine-grained Sketch-based Image Retrieval (2023)0.00
• Towards Unsupervised Sketch-based Image Retrieval (2021)0.00
• Adaptive Fine-grained Sketch-based Image Retrieval (2022)9.76
• Cross-modal Subspace Learning For Fine-grained Sketch-based Image Retrieval (2017)13.34
• Deep Reinforced Attention Regression For Partial Sketch Based Image Retrieval (2021)5.24
• A Recipe For Efficient SBIR Models: Combining Relative Triplet Loss With Batch Normalization And Knowledge Distillation (2023)0.00