Cross-modal Retrieval In The Cooking Context: Learning Semantic Text-image Embeddings

Abstract

Designing powerful tools that support cooking activities has rapidly gained popularity due to the massive amounts of available data, as well as recent advances in machine learning that are capable of analyzing them. In this paper, we propose a cross-modal retrieval model aligning visual and textual data (like pictures of dishes and their recipes) in a shared representation space. We describe an effective learning scheme, capable of tackling large-scale problems, and validate it on the Recipe1M dataset containing nearly 1 million picture-recipe pairs. We show the effectiveness of our approach regarding previous state-of-the-art models and present qualitative results over computational cooking use cases.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycarvalho2018cross

Related papers

• Images & Recipes: Retrieval In The Cooking Context (2018)3.58
• Cross-modal Food Retrieval: Learning A Joint Embedding Of Food Images And Recipes With Semantic Consistency And Attention Mechanism (2020)12.10
• Recipe1m+: A Dataset For Learning Cross-modal Embeddings For Cooking Recipes And Food Images (2018)17.24
• CHEF: Cross-modal Hierarchical Embeddings For Food Domain Retrieval (2021)8.35
• SIMMER: Cross-modal Food Image--recipe Retrieval Via Mllm-based Embedding (2026)0.00
• MCEN: Bridging Cross-modal Gap Between Cooking Recipes And Dish Images With Latent Variable Model (2020)13.39
• Mitigating Cross-modal Representation Bias For Multicultural Image-to-recipe Retrieval (2025)0.00
• Cross-modal Retrieval And Synthesis (X-MRS): Closing The Modality Gap In Shared Representation Learning (2020)0.00