Abstract
arXiv:2510.21151v2 Announce Type: replace Abstract: Multimodal conversational recommendation has recently emerged as a promising paradigm for delivering personalized experiences through natural dialogue enriched by visual and contextual grounding. Yet currently available multimodal conversational recommendation datasets remain limited: existing resources either simulate conversations, omit user history or fail to collect sufficiently detailed feedback, which constrain the types of research and evaluation they support. To address these gaps we introduce VOGUE, a dataset of 60 human human dialogues containing 2100 granularly labeled utterances in realistic fashion shopping scenarios. Each dialogue is paired with a shared visual catalogue, item metadata, user fashion profiles and post conversation ratings from both users (Seekers) and recommenders (Assistants). This design enables rigorous evaluation of conversational inference, including not only alignment between predicted and ground truth preferences but also calibration against full rating distributions and comparison with explicit and implicit user satisfaction signals. Our analyses of VOGUE reveal distinctive dynamics of visually grounded dialogue, e.g. recommenders frequently recommend items simultaneously in feature based groups, which creates distinct conversational phases bridged by Seeker critiques and refinements. Benchmarking Multimodal Large Language Models against human Recommenders shows that while MLLMs approach human level alignment in aggregate they exhibit systematic distribution errors in reproducing human ratings and struggle to generalize preference inference beyond explicitly discussed items. These findings establish VOGUE as both a unique resource for studying multimodal conversational systems and a challenge dataset beyond the current recommendation capabilities of existing top tier multimodal foundation models such as GPT-5-mini and Gemini-2.5-Flash.