Deep Reinforced Attention Regression For Partial Sketch Based Image Retrieval

Abstract

Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) aims at finding a specific image from a large gallery given a query sketch. Despite the widespread applicability of FG-SBIR in many critical domains (e.g., crime activity tracking), existing approaches still suffer from a low accuracy while being sensitive to external noises such as unnecessary strokes in the sketch. The retrieval performance will further deteriorate under a more practical on-the-fly setting, where only a partially complete sketch with only a few (noisy) strokes are available to retrieve corresponding images. We propose a novel framework that leverages a uniquely designed deep reinforcement learning model that performs a dual-level exploration to deal with partial sketch training and attention region selection. By enforcing the model's attention on the important regions of the original sketches, it remains robust to unnecessary stroke noises and improve the retrieval accuracy by a large margin. To sufficiently explore

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keywang2021deep

Related papers

• Sketch Less For More: On-the-fly Fine-grained Sketch Based Image Retrieval (2020)15.28
• Adaptive Fine-grained Sketch-based Image Retrieval (2022)9.76
• Active Learning For Fine-grained Sketch-based Image Retrieval (2023)0.00
• More Photos Are All You Need: Semi-supervised Learning For Fine-grained Sketch Based Image Retrieval (2021)13.23
• Cross-modal Hierarchical Modelling For Fine-grained Sketch Based Image Retrieval (2020)6.77
• Exploiting Unlabelled Photos For Stronger Fine-grained SBIR (2023)10.61
• A Zero-shot Framework For Sketch-based Image Retrieval (2018)16.49
• Arnet: Self-supervised FG-SBIR With Unified Sample Feature Alignment And Multi-scale Token Recycling (2024)5.84