Transformer-empowered Multi-modal Item Embedding For Enhanced Image Search In E-commerce

Abstract

Over the past decade, significant advances have been made in the field of image search for e-commerce applications. Traditional image-to-image retrieval models, which focus solely on image details such as texture, tend to overlook useful semantic information contained within the images. As a result, the retrieved products might possess similar image details, but fail to fulfil the user's search goals. Moreover, the use of image-to-image retrieval models for products containing multiple images results in significant online product feature storage overhead and complex mapping implementations. In this paper, we report the design and deployment of the proposed Multi-modal Item Embedding Model (MIEM) to address these limitations. It is capable of utilizing both textual information and multiple images about a product to construct meaningful product features. By leveraging semantic information from images, MIEM effectively supplements the image search process, improving the overall accuracy o

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyliu2023transformer

Related papers

• Optimizing Product Deduplication In E-commerce With Multimodal Embeddings (2025)0.00
• MRSE: An Efficient Multi-modality Retrieval System For Large Scale E-commerce (2024)0.00
• Multimodal Semantic Retrieval For Product Search (2025)3.58
• Semantic-enhanced Modality-asymmetric Retrieval For Online E-commerce Search (2025)0.00
• Unified Embedding Based Personalized Retrieval In Etsy Search (2023)2.26
• ACE-BERT: Adversarial Cross-modal Enhanced BERT For E-commerce Retrieval (2021)0.00
• Uniecs: Unified Multimodal E-commerce Search Framework With Gated Cross-modal Fusion (2025)2.60
• Embedding-based Product Retrieval In Taobao Search (2021)13.70