If At First You Don't Succeed: Test Time Re-ranking For Zero-shot, Cross-domain Retrieval

Abstract

In this paper, we introduce a novel method for zero-shot, cross-domain image retrieval. Our key contribution is a test-time Iterative Cluster-free Re-ranking process that leverages gallery-gallery feature information to establish semantic links between query and gallery images. This enables the retrieval of relevant images even when they do not exhibit similar visual features but share underlying semantic concepts. This can be combined with any pre-existing cross-domain feature extraction backbone to improve retrieval performance. However, when combined with a carefully chosen Vision Transformer backbone and combination of zero-shot retrieval losses, our approach yields state-of-the-art results on the Sketchy, TU-Berlin and QuickDraw sketch-based retrieval benchmarks. We show that our re-ranking also improves performance with other backbones and outperforms other re-ranking methods applied with our backbone. Importantly, unlike many previous methods, none of the components in our appro

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhudson2023if

Related papers

• Distribution Aligned Feature Clustering For Zero-shot Sketch-based Image Retrieval (2023)0.00
• Doodle To Search: Practical Zero-shot Sketch-based Image Retrieval (2019)16.75
• An Efficient Framework For Zero-shot Sketch-based Image Retrieval (2021)13.65
• Zero-shot Sketch Based Image Retrieval Using Graph Transformer (2022)6.77
• Zero-shot Everything Sketch-based Image Retrieval, And In Explainable Style (2023)16.67
• CLIP For All Things Zero-shot Sketch-based Image Retrieval, Fine-grained Or Not (2023)15.54
• Training-free Zero-shot Composed Image Retrieval With Local Concept Reranking (2023)0.00
• Towards Zero-shot Cross-lingual Image Retrieval (2020)2.46