For Manifold Learning, Deep Neural Networks Can Be Locality Sensitive Hash Functions

Abstract

It is well established that training deep neural networks gives useful representations that capture essential features of the inputs. However, these representations are poorly understood in theory and practice. In the context of supervised learning an important question is whether these representations capture features informative for classification, while filtering out non-informative noisy ones. We explore a formalization of this question by considering a generative process where each class is associated with a high-dimensional manifold and different classes define different manifolds. Under this model, each input is produced using two latent vectors: (i) a "manifold identifier" \(\gamma\) and; (ii)~a "transformation parameter" \(\theta\) that shifts examples along the surface of a manifold. E.g., \(\gamma\) might represent a canonical image of a dog, and \(\theta\) might stand for variations in pose, background or lighting. We provide theoretical and empirical evidence that neural r

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydikkala2021for

Related papers

• Randomly Weighted Neuromodulation In Neural Networks Facilitates Learning Of Manifolds Common Across Tasks (2023)0.00
• Mining On Manifolds: Metric Learning Without Labels (2018)14.31
• Connecting Neural Models Latent Geometries With Relative Geodesic Representations (2025)0.00
• Piecewise-linear Manifolds For Deep Metric Learning (2024)0.00
• Deepdiffusion: Unsupervised Learning Of Retrieval-adapted Representations Via Diffusion-based Ranking On Latent Feature Manifold (2021)5.13
• Latent Functional Maps: A Spectral Framework For Representation Alignment (2024)3.58
• Representing Deep Neural Networks Latent Space Geometries With Graphs (2020)7.50
• Metric Learning On Manifolds (2019)0.00