Training Vision Transformers For Image Retrieval

Abstract

Transformers have shown outstanding results for natural language understanding and, more recently, for image classification. We here extend this work and propose a transformer-based approach for image retrieval: we adopt vision transformers for generating image descriptors and train the resulting model with a metric learning objective, which combines a contrastive loss with a differential entropy regularizer. Our results show consistent and significant improvements of transformers over convolution-based approaches. In particular, our method outperforms the state of the art on several public benchmarks for category-level retrieval, namely Stanford Online Product, In-Shop and CUB-200. Furthermore, our experiments on ROxford and RParis also show that, in comparable settings, transformers are competitive for particular object retrieval, especially in the regime of short vector representations and low-resolution images.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyelnouby2021training

Related papers

• Boosting Vision Transformers For Image Retrieval (2022)15.28
• Investigating The Vision Transformer Model For Image Retrieval Tasks (2021)10.74
• Instance-level Image Retrieval Using Reranking Transformers (2021)19.00
• Thinking Fast And Slow: Efficient Text-to-visual Retrieval With Transformers (2021)15.16
• Vision Transformer Hashing For Image Retrieval (2021)17.01
• STIR: Siamese Transformer For Image Retrieval Postprocessing (2023)11.23
• Evidential Transformers For Improved Image Retrieval (2024)0.00
• Transmatcher: Deep Image Matching Through Transformers For Generalizable Person Re-identification (2021)4.68