Efficient Similarity-preserving Unsupervised Learning Using Modular Sparse Distributed Codes And Novelty-contingent Noise

Abstract

There is increasing realization in neuroscience that information is represented in the brain, e.g., neocortex, hippocampus, in the form sparse distributed codes (SDCs), a kind of cell assembly. Two essential questions are: a) how are such codes formed on the basis of single trials, and how is similarity preserved during learning, i.e., how do more similar inputs get mapped to more similar SDCs. I describe a novel Modular Sparse Distributed Code (MSDC) that provides simple, neurally plausible answers to both questions. An MSDC coding field (CF) consists of Q WTA competitive modules (CMs), each comprised of K binary units (analogs of principal cells). The modular nature of the CF makes possible a single-trial, unsupervised learning algorithm that approximately preserves similarity and crucially, runs in fixed time, i.e., the number of steps needed to store an item remains constant as the number of stored items grows. Further, once items are stored as MSDCs in superposition and such that

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrinkus2020efficient

Related papers

• Sparsey: Event Recognition Via Deep Hierarchical Spare Distributed Codes (2016)8.35
• Super-sparse Learning In Similarity Spaces (2017)5.24
• Unsupervised Representation Learning Via Neural Activation Coding (2021)1.56
• Nearest Neighbor Search With Compact Codes: A Decoder Perspective (2021)3.58
• Privacy-preserving Near Neighbor Search Via Sparse Coding With Ambiguation (2021)3.58
• Efficient And Reliable Vector Similarity Search Using Asymmetric Encoding With Nand-flash For Many-class Few-shot Learning (2024)2.26
• Unsupervised Deep Cross-modality Spectral Hashing (2020)11.39
• A New Similarity Space Tailored For Supervised Deep Metric Learning (2020)3.58