Contextclip: Contextual Alignment Of Image-text Pairs On CLIP Visual Representations

Abstract

State-of-the-art empirical work has shown that visual representations learned by deep neural networks are robust in nature and capable of performing classification tasks on diverse datasets. For example, CLIP demonstrated zero-shot transfer performance on multiple datasets for classification tasks in a joint embedding space of image and text pairs. However, it showed negative transfer performance on standard datasets, e.g., BirdsNAP, RESISC45, and MNIST. In this paper, we propose ContextCLIP, a contextual and contrastive learning framework for the contextual alignment of image-text pairs by learning robust visual representations on Conceptual Captions dataset. Our framework was observed to improve the image-text alignment by aligning text and image representations contextually in the joint embedding space. ContextCLIP showed good qualitative performance for text-to-image retrieval tasks and enhanced classification accuracy. We evaluated our model quantitatively with zero-shot transfer

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keygrover2022contextclip

Related papers

• Optimizing CLIP Models For Image Retrieval With Maintained Joint-embedding Alignment (2024)6.34
• \(\beta\)-clip: Text-conditioned Contrastive Learning For Multi-granular Vision-language Alignment (2025)2.16
• FG-CLIP: Fine-grained Visual And Textual Alignment (2025)5.75
• Clip-lite: Information Efficient Visual Representation Learning With Language Supervision (2021)2.35
• Medclip: Contrastive Learning From Unpaired Medical Images And Text (2022)26.02
• C-CLIP: Contrastive Image-text Encoders To Close The Descriptive-commentative Gap (2023)0.00
• Superclip: CLIP With Simple Classification Supervision (2025)0.00
• Linear Alignment Of Vision-language Models For Image Captioning (2023)0.00