Self-calibrating Neural Networks For Dimensionality Reduction

Abstract

Recently, a novel family of biologically plausible online algorithms for reducing the dimensionality of streaming data has been derived from the similarity matching principle. In these algorithms, the number of output dimensions can be determined adaptively by thresholding the singular values of the input data matrix. However, setting such threshold requires knowing the magnitude of the desired singular values in advance. Here we propose online algorithms where the threshold is self-calibrating based on the singular values computed from the existing observations. To derive these algorithms from the similarity matching cost function we propose novel regularizers. As before, these online algorithms can be implemented by Hebbian/anti-Hebbian neural networks in which the learning rule depends on the chosen regularizer. We demonstrate both mathematically and via simulation the effectiveness of these online algorithms in various settings.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2016self

Related papers

• TLDR: Twin Learning For Dimensionality Reduction (2021)0.00
• Kernel Similarity Matching With Hebbian Neural Networks (2022)0.00
• Learning-based Dimensionality Reduction For Computing Compact And Effective Local Feature Descriptors (2022)9.99
• Dimensionality-reduction Techniques For Approximate Nearest Neighbor Search: A Survey And Evaluation (2024)0.00
• Neural Nearest Neighbors Networks (2018)0.00
• Interactive Dimensionality Reduction Using Similarity Projections (2018)8.09
• Sparse Online Relative Similarity Learning (2021)2.26
• Gleanvec: Accelerating Vector Search With Minimalist Nonlinear Dimensionality Reduction (2024)0.00