Semantically Constrained Memory Allocation (SCMA) For Embedding In Efficient Recommendation Systems

Abstract

Deep learning-based models are utilized to achieve state-of-the-art performance for recommendation systems. A key challenge for these models is to work with millions of categorical classes or tokens. The standard approach is to learn end-to-end, dense latent representations or embeddings for each token. The resulting embeddings require large amounts of memory that blow up with the number of tokens. Training and inference with these models create storage, and memory bandwidth bottlenecks leading to significant computing and energy consumption when deployed in practice. To this end, we present the problem of \textit\{Memory Allocation\} under budget for embeddings and propose a novel formulation of memory shared embedding, where memory is shared in proportion to the overlap in semantic information. Our formulation admits a practical and efficient randomized solution with Locality sensitive hashing based Memory Allocation (LMA). We demonstrate a significant reduction in the memory footpri

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydesai2021semantically

Related papers

• Mem-rec: Memory Efficient Recommendation System Using Alternative Representation (2023)0.00
• CAFE: Towards Compact, Adaptive, And Fast Embedding For Large-scale Recommendation Models (2023)8.09
• Mixed-precision Embeddings For Large-scale Recommendation Models (2024)0.00
• Learning To Collide: Recommendation System Model Compression With Learned Hash Functions (2022)0.00
• SMEC: Rethinking Matryoshka Representation Learning For Retrieval Embedding Compression (2025)0.00
• Learning Compact Compositional Embeddings Via Regularized Pruning For Recommendation (2023)8.36
• Saec: Similarity-aware Embedding Compression In Recommendation Systems (2019)0.00
• Learning Compressed Embeddings For On-device Inference (2022)0.00