Deep Image Representations Using Caption Generators

Abstract

Deep learning exploits large volumes of labeled data to learn powerful models. When the target dataset is small, it is a common practice to perform transfer learning using pre-trained models to learn new task specific representations. However, pre-trained CNNs for image recognition are provided with limited information about the image during training, which is label alone. Tasks such as scene retrieval suffer from features learned from this weak supervision and require stronger supervision to better understand the contents of the image. In this paper, we exploit the features learned from caption generating models to learn novel task specific image representations. In particular, we consider the state-of-the art captioning system Show and Tell~\cite\{SnT-pami-2016\} and the dense region description model DenseCap~\cite\{densecap-cvpr-2016\}. We demonstrate that, owing to richer supervision provided during the process of training, the features learned by the captioning system perform bet

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymopuri2017deep

Related papers

• Show, Translate And Tell (2019)4.52
• Towards Retrieval-augmented Architectures For Image Captioning (2024)9.41
• Retrieval-augmented Image Captioning (2023)11.29
• Distinctive Image Captioning: Leveraging Ground Truth Captions In CLIP Guided Reinforcement Learning (2024)4.52
• Dualcap: Enhancing Lightweight Image Captioning Via Dual Retrieval With Similar Scenes Visual Prompts (2025)0.00
• Dreamlip: Language-image Pre-training With Long Captions (2024)10.61
• Retrieval Augmentation For Deep Neural Networks (2021)5.84
• Learning Visually Grounded Sentence Representations (2017)7.81