Cross-modal Attention Alignment Network With Auxiliary Text Description For Zero-shot Sketch-based Image Retrieval

Abstract

In this paper, we study the problem of zero-shot sketch-based image retrieval (ZS-SBIR). The prior methods tackle the problem in a two-modality setting with only category labels or even no textual information involved. However, the growing prevalence of Large-scale pre-trained Language Models (LLMs), which have demonstrated great knowledge learned from web-scale data, can provide us with an opportunity to conclude collective textual information. Our key innovation lies in the usage of text data as auxiliary information for images, thus leveraging the inherent zero-shot generalization ability that language offers. To this end, we propose an approach called Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval. The network consists of three components: (i) a Description Generation Module that generates textual descriptions for each training category by prompting an LLM with several interrogative sentences, (ii) a Feature Extrac

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keysu2024cross

Related papers

• Semantic Adversarial Network For Zero-shot Sketch-based Image Retrieval (2019)10.74
• Adapt And Align To Improve Zero-shot Sketch-based Image Retrieval (2023)0.00
• Zero-shot Everything Sketch-based Image Retrieval, And In Explainable Style (2023)16.67
• Dynamic Multi-level Weighted Alignment Network For Zero-shot Sketch-based Image Retrieval (2025)0.00
• Modality-aware Representation Learning For Zero-shot Sketch-based Image Retrieval (2024)8.60
• Stacked Semantic-guided Network For Zero-shot Sketch-based Image Retrieval (2019)0.00
• Crossatnet - A Novel Cross-attention Based Framework For Sketch-based Image Retrieval (2021)11.29
• Domain-smoothing Network For Zero-shot Sketch-based Image Retrieval (2021)13.92