Sketch3t: Test-time Training For Zero-shot SBIR
2022 · Aneeshan Sain, Ayan Kumar Bhunia, Vaishnav Potlapalli, et al.
Abstract
Zero-shot sketch-based image retrieval typically asks for a trained model to be applied as is to unseen categories. In this paper, we question to argue that this setup by definition is not compatible with the inherent abstract and subjective nature of sketches, i.e., the model might transfer well to new categories, but will not understand sketches existing in different test-time distribution as a result. We thus extend ZS-SBIR asking it to transfer to both categories and sketch distributions. Our key contribution is a test-time training paradigm that can adapt using just one sketch. Since there is no paired photo, we make use of a sketch raster-vector reconstruction module as a self-supervised auxiliary task. To maintain the fidelity of the trained cross-modal joint embedding during test-time update, we design a novel meta-learning based training paradigm to learn a separation between model updates incurred by this auxiliary task from those off the primary objective of discriminative l
Authors
(none)
Tags
Stats
Related papers
- Relation-aware Meta-learning For Zero-shot Sketch-based Image Retrieval (2024)0.00
- A Zero-shot Framework For Sketch-based Image Retrieval (2018)16.49
- Semantic Adversarial Network For Zero-shot Sketch-based Image Retrieval (2019)10.74
- Generative Model For Zero-shot Sketch-based Image Retrieval (2019)9.23
- Doodle To Search: Practical Zero-shot Sketch-based Image Retrieval (2019)16.75
- An Efficient Framework For Zero-shot Sketch-based Image Retrieval (2021)13.65
- Zero-shot Everything Sketch-based Image Retrieval, And In Explainable Style (2023)16.67
- CLIP For All Things Zero-shot Sketch-based Image Retrieval, Fine-grained Or Not (2023)15.54