Semi-heterogeneous Three-way Joint Embedding Network For Sketch-based Image Retrieval

Abstract

Sketch-based image retrieval (SBIR) is a challenging task due to the large cross-domain gap between sketches and natural images. How to align abstract sketches and natural images into a common high-level semantic space remains a key problem in SBIR. In this paper, we propose a novel semi-heterogeneous three-way joint embedding network (Semi3-Net), which integrates three branches (a sketch branch, a natural image branch, and an edgemap branch) to learn more discriminative cross-domain feature representations for the SBIR task. The key insight lies with how we cultivate the mutual and subtle relationships amongst the sketches, natural images, and edgemaps. A semi-heterogeneous feature mapping is designed to extract bottom features from each domain, where the sketch and edgemap branches are shared while the natural image branch is heterogeneous to the other branches. In addition, a joint semantic embedding is introduced to embed the features from different domains into a common high-level

Stats

• citations45
• S2 citations—
• github stars0
• HF likes0
• heat score12.47
• arxiv keylei2019semi

Related papers

• Three-stream Joint Network For Zero-shot Sketch-based Image Retrieval (2022)0.00
• Cross-modal Subspace Learning For Fine-grained Sketch-based Image Retrieval (2017)13.34
• Stacked Semantic-guided Network For Zero-shot Sketch-based Image Retrieval (2019)0.00
• Towards Unsupervised Sketch-based Image Retrieval (2021)0.00
• Domain-aware SE Network For Sketch-based Image Retrieval With Multiplicative Euclidean Margin Softmax (2018)8.09
• Relation-aware Meta-learning For Zero-shot Sketch-based Image Retrieval (2024)0.00
• Semantically Tied Paired Cycle Consistency For Zero-shot Sketch-based Image Retrieval (2019)15.70
• Semantic Adversarial Network For Zero-shot Sketch-based Image Retrieval (2019)10.74