Zero-shot Sketch Based Image Retrieval Using Graph Transformer

Abstract

The performance of a zero-shot sketch-based image retrieval (ZS-SBIR) task is primarily affected by two challenges. The substantial domain gap between image and sketch features needs to be bridged, while at the same time the side information has to be chosen tactfully. Existing literature has shown that varying the semantic side information greatly affects the performance of ZS-SBIR. To this end, we propose a novel graph transformer based zero-shot sketch-based image retrieval (GTZSR) framework for solving ZS-SBIR tasks which uses a novel graph transformer to preserve the topology of the classes in the semantic space and propagates the context-graph of the classes within the embedding features of the visual space. To bridge the domain gap between the visual features, we propose minimizing the Wasserstein distance between images and sketches in a learned domain-shared space. We also propose a novel compatibility loss that further aligns the two visual domains by bridging the domain gap

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keygupta2022zero

Related papers

• Stacked Semantic-guided Network For Zero-shot Sketch-based Image Retrieval (2019)0.00
• Semantic Adversarial Network For Zero-shot Sketch-based Image Retrieval (2019)10.74
• An Efficient Framework For Zero-shot Sketch-based Image Retrieval (2021)13.65
• Adapt And Align To Improve Zero-shot Sketch-based Image Retrieval (2023)0.00
• Domain-smoothing Network For Zero-shot Sketch-based Image Retrieval (2021)13.92
• Relation-aware Meta-learning For Zero-shot Sketch-based Image Retrieval (2024)0.00
• Stacked Adversarial Network For Zero-shot Sketch Based Image Retrieval (2020)10.74
• Zero-shot Everything Sketch-based Image Retrieval, And In Explainable Style (2023)16.67