Grafit: Learning Fine-grained Image Representations With Coarse Labels

Abstract

This paper tackles the problem of learning a finer representation than the one provided by training labels. This enables fine-grained category retrieval of images in a collection annotated with coarse labels only. Our network is learned with a nearest-neighbor classifier objective, and an instance loss inspired by self-supervised learning. By jointly leveraging the coarse labels and the underlying fine-grained latent space, it significantly improves the accuracy of category-level retrieval methods. Our strategy outperforms all competing methods for retrieving or classifying images at a finer granularity than that available at train time. It also improves the accuracy for transfer learning tasks to fine-grained datasets, thereby establishing the new state of the art on five public benchmarks, like iNaturalist-2018.

Stats

• citations52
• S2 citations—
• github stars0
• HF likes0
• heat score12.93
• arxiv keytouvron2020grafit

Related papers

• The Curious Layperson: Fine-grained Image Recognition Without Expert Labels (2021)9.99
• Re-rank Coarse Classification With Local Region Enhanced Features For Fine-grained Image Recognition (2021)0.00
• Coarse-to-fine: Learning Compact Discriminative Representation For Single-stage Image Retrieval (2023)9.35
• Selective Convolutional Descriptor Aggregation For Fine-grained Image Retrieval (2016)19.37
• Exploiting Unlabelled Photos For Stronger Fine-grained SBIR (2023)10.61
• Self-supervising Fine-grained Region Similarities For Large-scale Image Localization (2020)15.85
• Fine-tuning CNN Image Retrieval With No Human Annotation (2017)22.75
• Multi-modal Reference Learning For Fine-grained Text-to-image Retrieval (2025)6.77