2-D Embedding Of Large And High-dimensional Data With Minimal Memory And Computational Time Requirements

Abstract

In the advent of big data era, interactive visualization of large data sets consisting of M*10^5+ high-dimensional feature vectors of length N (N ~ 10^3+), is an indispensable tool for data exploratory analysis. The state-of-the-art data embedding (DE) methods of N-D data into 2-D (3-D) visually perceptible space (e.g., based on t-SNE concept) are too demanding computationally to be efficiently employed for interactive data analytics of large and high-dimensional datasets. Herein we present a simple method, ivhd (interactive visualization of high-dimensional data tool), which radically outperforms the modern data-embedding algorithms in both computational and memory loads, while retaining high quality of N-D data embedding in 2-D (3-D). We show that DE problem is equivalent to the nearest neighbor nn-graph visualization, where only indices of a few nearest neighbors of each data sample has to be known, and binary distance between data samples -- 0 to the nearest and 1 to the other samp

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydzwinel20192

Related papers

• In Search Of The Most Efficient And Memory-saving Visualization Of High Dimensional Data (2023)0.00
• Visualizing Large-scale And High-dimensional Data (2016)18.48
• Interactive Dimensionality Reduction Using Similarity Projections (2018)8.09
• Exploring The Meaningfulness Of Nearest Neighbor Search In High-dimensional Space (2024)2.26
• A Shallow High-order Parametric Approach To Data Visualization And Compression (2016)0.00
• Hd-index: Pushing The Scalability-accuracy Boundary For Approximate Knn Search In High-dimensional Spaces (2018)14.02
• Gleanvec: Accelerating Vector Search With Minimalist Nonlinear Dimensionality Reduction (2024)0.00
• Efficient Data-aware Distance Comparison Operations For High-dimensional Approximate Nearest Neighbor Search (2024)5.24