Semantically Tied Paired Cycle Consistency For Zero-shot Sketch-based Image Retrieval

Abstract

Zero-shot sketch-based image retrieval (SBIR) is an emerging task in computer vision, allowing to retrieve natural images relevant to sketch queries that might not been seen in the training phase. Existing works either require aligned sketch-image pairs or inefficient memory fusion layer for mapping the visual information to a semantic space. In this work, we propose a semantically aligned paired cycle-consistent generative (SEM-PCYC) model for zero-shot SBIR, where each branch maps the visual information to a common semantic space via an adversarial training. Each of these branches maintains a cycle consistency that only requires supervision at category levels, and avoids the need of highly-priced aligned sketch-image pairs. A classification criteria on the generators' outputs ensures the visual to semantic space mapping to be discriminating. Furthermore, we propose to combine textual and hierarchical side information via a feature selection auto-encoder that selects discriminating si

Stats

• citations123
• S2 citations—
• github stars0
• HF likes0
• heat score15.70
• arxiv keydutta2019semantically

Related papers

• Semantically Tied Paired Cycle Consistency For Any-shot Sketch-based Image Retrieval (2020)10.61
• Semantic Adversarial Network For Zero-shot Sketch-based Image Retrieval (2019)10.74
• Relation-aware Meta-learning For Zero-shot Sketch-based Image Retrieval (2024)0.00
• Stacked Semantic-guided Network For Zero-shot Sketch-based Image Retrieval (2019)0.00
• A Zero-shot Framework For Sketch-based Image Retrieval (2018)16.49
• Generative Model For Zero-shot Sketch-based Image Retrieval (2019)9.23
• Stacked Adversarial Network For Zero-shot Sketch Based Image Retrieval (2020)10.74
• An Efficient Framework For Zero-shot Sketch-based Image Retrieval (2021)13.65