Contrastive Learning With Stronger Augmentations

Abstract

Representation learning has significantly been developed with the advance of contrastive learning methods. Most of those methods have benefited from various data augmentations that are carefully designated to maintain their identities so that the images transformed from the same instance can still be retrieved. However, those carefully designed transformations limited us to further explore the novel patterns exposed by other transformations. Meanwhile, as found in our experiments, the strong augmentations distorted the images' structures, resulting in difficult retrieval. Thus, we propose a general framework called Contrastive Learning with Stronger Augmentations~(CLSA) to complement current contrastive learning approaches. Here, the distribution divergence between the weakly and strongly augmented images over the representation bank is adopted to supervise the retrieval of strongly augmented queries from a pool of instances. Experiments on the ImageNet dataset and downstream datasets

Stats

• citations136
• S2 citations—
• github stars61
• HF likes0
• heat score19.61
• arxiv keywang2021contrastive

Related papers

• Robust Cross-modal Representation Learning With Progressive Self-distillation (2022)12.33
• Augnet: End-to-end Unsupervised Visual Representation Learning With Image Augmentation (2021)3.96
• Improving Composed Image Retrieval Via Contrastive Learning With Scaling Positives And Negatives (2024)11.30
• With A Little Help From My Friends: Nearest-neighbor Contrastive Learning Of Visual Representations (2021)18.76
• A Review Of Image Retrieval Techniques: Data Augmentation And Adversarial Learning Approaches (2024)0.00
• Learning Customized Visual Models With Retrieval-augmented Knowledge (2023)11.58
• Linking Representations With Multimodal Contrastive Learning (2023)0.00
• Revisiting Contrastive Methods For Unsupervised Learning Of Visual Representations (2021)3.91