Curator: Creating Large-scale Curated Labelled Datasets Using Self-supervised Learning

Abstract

Applying Machine learning to domains like Earth Sciences is impeded by the lack of labeled data, despite a large corpus of raw data available in such domains. For instance, training a wildfire classifier on satellite imagery requires curating a massive and diverse dataset, which is an expensive and time-consuming process that can span from weeks to months. Searching for relevant examples in over 40 petabytes of unlabelled data requires researchers to manually hunt for such images, much like finding a needle in a haystack. We present a no-code end-to-end pipeline, Curator, which dramatically minimizes the time taken to curate an exhaustive labeled dataset. Curator is able to search massive amounts of unlabelled data by combining self-supervision, scalable nearest neighbor search, and active learning to learn and differentiate image representations. The pipeline can also be readily applied to solve problems across different domains. Overall, the pipeline makes it practical for researcher

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynarayanan2022curator

Related papers

• Unsupervised Feature Learning In Remote Sensing (2019)4.52
• Learning More From Less: Towards Strengthening Weak Supervision For Ad-hoc Retrieval (2019)5.84
• Improving Spatiotemporal Self-supervision By Deep Reinforcement Learning (2018)13.50
• Mlrsnet: A Multi-label High Spatial Resolution Remote Sensing Dataset For Semantic Scene Understanding (2020)19.63
• Visual Representation Learning With Self-supervised Attention For Low-label High-data Regime (2022)5.49
• Learning To Learn In A Semi-supervised Fashion (2020)2.26
• Self-supervising Fine-grained Region Similarities For Large-scale Image Localization (2020)15.85
• SEPT: Towards Scalable And Efficient Visual Pre-training (2022)3.58