Arnet: Self-supervised FG-SBIR With Unified Sample Feature Alignment And Multi-scale Token Recycling

Abstract

Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) aims to minimize the distance between sketches and corresponding images in the embedding space. However, scalability is hindered by the growing complexity of solutions, mainly due to the abstract nature of fine-grained sketches. In this paper, we propose an effective approach to narrow the gap between the two domains. It mainly facilitates unified mutual information sharing both intra- and inter-samples, rather than treating them as a single feature alignment problem between modalities. Specifically, our approach includes: (i) Employing dual weight-sharing networks to optimize alignment within the sketch and image domain, which also effectively mitigates model learning saturation issues. (ii) Introducing an objective optimization function based on contrastive loss to enhance the model's ability to align features in both intra- and inter-samples. (iii) Presenting a self-supervised Multi-Scale Token Recycling (MSTR) Module featured by r

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyjiang2024arnet

Related papers

• Adaptive Fine-grained Sketch-based Image Retrieval (2022)9.76
• Dynamic Multi-level Weighted Alignment Network For Zero-shot Sketch-based Image Retrieval (2025)0.00
• Deep Reinforced Attention Regression For Partial Sketch Based Image Retrieval (2021)5.24
• Towards Unsupervised Sketch-based Image Retrieval (2021)0.00
• Exploiting Unlabelled Photos For Stronger Fine-grained SBIR (2023)10.61
• Sketch Less For More: On-the-fly Fine-grained Sketch Based Image Retrieval (2020)15.28
• More Photos Are All You Need: Semi-supervised Learning For Fine-grained Sketch Based Image Retrieval (2021)13.23
• Semi-heterogeneous Three-way Joint Embedding Network For Sketch-based Image Retrieval (2019)12.47