RECE: Reduced Cross-entropy Loss For Large-catalogue Sequential Recommenders

Abstract

Scalability is a major challenge in modern recommender systems. In sequential recommendations, full Cross-Entropy (CE) loss achieves state-of-the-art recommendation quality but consumes excessive GPU memory with large item catalogs, limiting its practicality. Using a GPU-efficient locality-sensitive hashing-like algorithm for approximating large tensor of logits, this paper introduces a novel RECE (REduced Cross-Entropy) loss. RECE significantly reduces memory consumption while allowing one to enjoy the state-of-the-art performance of full CE loss. Experimental results on various datasets show that RECE cuts training peak memory usage by up to 12 times compared to existing methods while retaining or exceeding performance metrics of CE loss. The approach also opens up new possibilities for large-scale applications in other domains.

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keygusak2024rece

Related papers

• Recjpq: Training Large-catalogue Sequential Recommenders (2023)8.09
• Learning Similarity Preserving Binary Codes For Recommender Systems (2022)0.00
• Mem-rec: Memory Efficient Recommendation System Using Alternative Representation (2023)0.00
• Croloss: Towards A Customizable Loss For Retrieval Models In Recommender Systems (2022)5.84
• Domain-adaptive And Scalable Dense Retrieval For Content-based Recommendation (2026)0.00
• Hierarchical Structured Neural Network: Efficient Retrieval Scaling For Large Scale Recommendation (2024)0.00
• Mixed-precision Embeddings For Large-scale Recommendation Models (2024)0.00
• Saec: Similarity-aware Embedding Compression In Recommendation Systems (2019)0.00