A Hybrid Multimodal Deep Learning Framework For Intelligent Fashion Recommendation

Abstract

The rapid expansion of online fashion platforms has created an increasing demand for intelligent recommender systems capable of understanding both visual and textual cues. This paper proposes a hybrid multimodal deep learning framework for fashion recommendation that jointly addresses two key tasks: outfit compatibility prediction and complementary item retrieval. The model leverages the visual and textual encoders of the CLIP architecture to obtain joint latent representations of fashion items, which are then integrated into a unified feature vector and processed by a transformer encoder. For compatibility prediction, an "outfit token" is introduced to model the holistic relationships among items, achieving an AUC of 0.95 on the Polyvore dataset. For complementary item retrieval, a "target item token" representing the desired item description is used to retrieve compatible items, reaching an accuracy of 69.24% under the Fill-in-the-Blank (FITB) metric. The proposed approach demonstrat

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykalashi2025a

Related papers

• Outfittransformer: Learning Outfit Representations For Fashion Recommendation (2022)10.61
• Fashion Outfit Complementary Item Retrieval (2019)14.06
• Image Based Fashion Product Recommendation With Deep Learning (2018)12.74
• Kisekloset For Fashion Retrieval And Recommendation (2025)0.00
• Fad-vlp: Fashion Vision-and-language Pre-training Towards Unified Retrieval And Captioning (2022)7.81
• Unifashion: A Unified Vision-language Model For Multimodal Fashion Retrieval And Generation (2024)10.66
• TATTOO: Training-free Aesthetic-aware Outfit Recommendation (2025)0.00
• Diversity In Fashion Recommendation Using Semantic Parsing (2019)10.21