Uniform Memory Retrieval With Larger Capacity For Modern Hopfield Models

Abstract

We propose a two-stage memory retrieval dynamics for modern Hopfield models, termed \(\mathtt\{U\text\{-\}Hop\}\), with enhanced memory capacity. Our key contribution is a learnable feature map \(\Phi\) which transforms the Hopfield energy function into kernel space. This transformation ensures convergence between the local minima of energy and the fixed points of retrieval dynamics within the kernel space. Consequently, the kernel norm induced by \(\Phi\) serves as a novel similarity measure. It utilizes the stored memory patterns as learning data to enhance memory capacity across all modern Hopfield models. Specifically, we accomplish this by constructing a separation loss \(\mathcal\{L\}_\Phi\) that separates the local minima of kernelized energy by separating stored memory patterns in kernel space. Methodologically, \(\mathtt\{U\text\{-\}Hop\}\) memory retrieval process consists of: (Stage I) minimizing separation loss for a more uniform memory (local minimum) distribution, followe

Stats

• citations0
• S2 citations—
• github stars9
• HF likes0
• heat score2.00
• arxiv keywu2024uniform

Related papers

• Unifier: A Unified Retriever For Large-scale Retrieval (2022)7.50
• Shuffle And Learn: Minimizing Mutual Information For Unsupervised Hashing (2020)0.00
• Breaking The Curse Of Dimensionality: On The Stability Of Modern Vector Retrieval (2025)0.00
• Hyp-uml: Hyperbolic Image Retrieval With Uncertainty-aware Metric Learning (2023)0.00
• Dual-tuning: Joint Prototype Transfer And Structure Regularization For Compatible Feature Learning (2021)6.07
• AMES: Asymmetric And Memory-efficient Similarity Estimation For Instance-level Retrieval (2024)9.70
• Phase Diagram Of Restricted Boltzmann Machines And Generalised Hopfield Networks With Arbitrary Priors (2017)14.02
• Kernel Similarity Matching With Hebbian Neural Networks (2022)0.00