Automatic Spatially-aware Fashion Concept Discovery

Abstract

This paper proposes an automatic spatially-aware concept discovery approach using weakly labeled image-text data from shopping websites. We first fine-tune GoogleNet by jointly modeling clothing images and their corresponding descriptions in a visual-semantic embedding space. Then, for each attribute (word), we generate its spatially-aware representation by combining its semantic word vector representation with its spatial representation derived from the convolutional maps of the fine-tuned network. The resulting spatially-aware representations are further used to cluster attributes into multiple groups to form spatially-aware concepts (e.g., the neckline concept might consist of attributes like v-neck, round-neck, etc). Finally, we decompose the visual-semantic embedding space into multiple concept-specific subspaces, which facilitates structured browsing and attribute-feedback product retrieval by exploiting multimodal linguistic regularities. We conducted extensive experiments on ou

Stats

• citations174
• S2 citations—
• github stars0
• HF likes0
• heat score16.82
• arxiv keyhan2017automatic

Related papers

• Snap And Find: Deep Discrete Cross-domain Garment Image Retrieval (2019)0.00
• Leveraging Weakly Annotated Data For Fashion Image Retrieval And Label Prediction (2017)14.06
• Fashion-specific Attributes Interpretation Via Dual Gaussian Visual-semantic Embedding (2022)0.00
• Fashionsearchnet-v2: Learning Attribute Representations With Localization For Image Retrieval With Attribute Manipulation (2021)0.00
• Searching For Apparel Products From Images In The Wild (2019)0.00
• Fine-grained Apparel Classification And Retrieval Without Rich Annotations (2018)0.00
• Mmfl-net: Multi-scale And Multi-granularity Feature Learning For Cross-domain Fashion Retrieval (2022)5.84
• Clothing Retrieval With Visual Attention Model (2017)12.10