Multi-probe Zero Collision Hash (MPZCH): Mitigating Embedding Collisions And Enhancing Model Freshness In Large-scale Recommenders

Abstract

Embedding tables are critical components of large-scale recommendation systems, facilitating the efficient mapping of high-cardinality categorical features into dense vector representations. However, as the volume of unique IDs expands, traditional hash-based indexing methods suffer from collisions that degrade model performance and personalization quality. We present Multi-Probe Zero Collision Hash (MPZCH), a novel indexing mechanism based on linear probing that effectively mitigates embedding collisions. With reasonable table sizing, it often eliminates these collisions entirely while maintaining production-scale efficiency. MPZCH utilizes auxiliary tensors and high-performance CUDA kernels to implement configurable probing and active eviction policies. By retiring obsolete IDs and resetting reassigned slots, MPZCH prevents the stale embedding inheritance typical of hash-based methods, ensuring new features learn effectively from scratch. Despite its collision-mitigation overhead, th

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhao2026multi

Related papers

• Learning To Collide: Recommendation System Model Compression With Learned Hash Functions (2022)0.00
• Mixed-precision Embeddings For Large-scale Recommendation Models (2024)0.00
• Graphhash: Graph Clustering Enables Parameter Efficiency In Recommender Systems (2024)4.77
• Mem-rec: Memory Efficient Recommendation System Using Alternative Representation (2023)0.00
• Multi-feature Discrete Collaborative Filtering For Fast Cold-start Recommendation (2020)9.59
• Learning To Hash For Recommendation: A Survey (2024)0.00
• Collaborative Group-aware Hashing For Fast Recommender Systems (2025)0.00
• Promips: Efficient High-dimensional C-approximate Maximum Inner Product Search With A Lightweight Index (2021)8.35