MRSE: An Efficient Multi-modality Retrieval System For Large Scale E-commerce

Abstract

Providing high-quality item recall for text queries is crucial in large-scale e-commerce search systems. Current Embedding-based Retrieval Systems (ERS) embed queries and items into a shared low-dimensional space, but uni-modality ERS rely too heavily on textual features, making them unreliable in complex contexts. While multi-modality ERS incorporate various data sources, they often overlook individual preferences for different modalities, leading to suboptimal results. To address these issues, we propose MRSE, a Multi-modality Retrieval System that integrates text, item images, and user preferences through lightweight mixture-of-expert (LMoE) modules to better align features across and within modalities. MRSE also builds user profiles at a multi-modality level and introduces a novel hybrid loss function that enhances consistency and robustness using hard negative sampling. Experiments on a large-scale dataset from Shopee and online A/B testing show that MRSE achieves an 18.9% improve

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjiang2024mrse

Related papers

• Semantic-enhanced Modality-asymmetric Retrieval For Online E-commerce Search (2025)0.00
• Transformer-empowered Multi-modal Item Embedding For Enhanced Image Search In E-commerce (2023)4.52
• Multimodal Semantic Retrieval For Product Search (2025)3.58
• Embedding-based Product Retrieval In Taobao Search (2021)13.70
• Multi-objective Personalized Product Retrieval In Taobao Search (2022)0.00
• Uniecs: Unified Multimodal E-commerce Search Framework With Gated Cross-modal Fusion (2025)2.60
• Large Reasoning Embedding Models: Towards Next-generation Dense Retrieval Paradigm (2025)0.00
• Unified Learning-to-rank For Multi-channel Retrieval In Large-scale E-commerce Search (2026)0.00