Kernel Similarity Matching With Hebbian Neural Networks

Abstract

Recent works have derived neural networks with online correlation-based learning rules to perform \textit\{kernel similarity matching\}. These works applied existing linear similarity matching algorithms to nonlinear features generated with random Fourier methods. In this paper attempt to perform kernel similarity matching by directly learning the nonlinear features. Our algorithm proceeds by deriving and then minimizing an upper bound for the sum of squared errors between output and input kernel similarities. The construction of our upper bound leads to online correlation-based learning rules which can be implemented with a 1 layer recurrent neural network. In addition to generating high-dimensional linearly separable representations, we show that our upper bound naturally yields representations which are sparse and selective for specific input patterns. We compare the approximation quality of our method to neural random Fourier method and variants of the popular but non-biological "N

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyluther2022kernel

Related papers

• Similarity Learning Via Kernel Preserving Embedding (2019)10.35
• Neural Nearest Neighbors Networks (2018)0.00
• Self-calibrating Neural Networks For Dimensionality Reduction (2016)0.00
• Scaling Up Kernel Ridge Regression Via Locality Sensitive Hashing (2020)0.00
• Efficient Optimization Methods For Extreme Similarity Learning With Nonlinear Embeddings (2020)3.58
• Learning Non-metric Visual Similarity For Image Retrieval (2017)11.58
• Stochastic Learning Of Nonstationary Kernels For Natural Language Modeling (2018)0.00
• Learning-based Efficient Graph Similarity Computation Via Multi-scale Convolutional Set Matching (2018)13.60