An Instance Selection Algorithm For Big Data In High Imbalanced Datasets Based On LSH

Abstract

Training of Machine Learning (ML) models in real contexts often deals with big data sets and high-class imbalance samples where the class of interest is unrepresented (minority class). Practical solutions using classical ML models address the problem of large data sets using parallel/distributed implementations of training algorithms, approximate model-based solutions, or applying instance selection (IS) algorithms to eliminate redundant information. However, the combined problem of big and high imbalanced datasets has been less addressed. This work proposes three new methods for IS to be able to deal with large and imbalanced data sets. The proposed methods use Locality Sensitive Hashing (LSH) as a base clustering technique, and then three different sampling methods are applied on top of the clusters (or buckets) generated by LSH. The algorithms were developed in the Apache Spark framework, guaranteeing their scalability. The experiments carried out in three different datasets suggest

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymeloacosta2022an

Related papers

• Scalable Graph Attention-based Instance Selection Via Mini-batch Sampling And Hierarchical Hashing (2025)0.00
• Experimental Analysis Of Machine Learning Techniques For Finding Search Radius In Locality Sensitive Hashing (2022)0.00
• Hierarchical Locality Sensitive Hashing For Structured Data: A Survey (2022)0.00
• Experimental Analysis Of Locality Sensitive Hashing Techniques For High-dimensional Approximate Nearest Neighbor Searches (2020)6.34
• Improving Locality Sensitive Hashing By Efficiently Finding Projected Nearest Neighbors (2020)6.77
• Drawbacks And Proposed Solutions For Real-time Processing On Existing State-of-the-art Locality Sensitive Hashing Techniques (2019)0.00
• Improving Similarity Search With High-dimensional Locality-sensitive Hashing (2018)0.00
• Locality-sensitive Hashing For F-divergences: Mutual Information Loss And Beyond (2019)0.00