Chatting Makes Perfect: Chat-based Image Retrieval
2023 · Matan Levy, Rami Ben-Ari, Nir Darshan, et al.
Abstract
Chats emerge as an effective user-friendly approach for information retrieval, and are successfully employed in many domains, such as customer service, healthcare, and finance. However, existing image retrieval approaches typically address the case of a single query-to-image round, and the use of chats for image retrieval has been mostly overlooked. In this work, we introduce ChatIR: a chat-based image retrieval system that engages in a conversation with the user to elicit information, in addition to an initial query, in order to clarify the user's search intent. Motivated by the capabilities of today's foundation models, we leverage Large Language Models to generate follow-up questions to an initial image description. These questions form a dialog with the user in order to retrieve the desired image from a large corpus. In this study, we explore the capabilities of such a system tested on a large dataset and reveal that engaging in a dialog yields significant gains in image retrieval.
Authors
(none)
Tags
Stats
Related papers
- Chatsearch: A Dataset And A Generative Retrieval Model For General Conversational Image Retrieval (2024)2.00
- Dialog-based Interactive Image Retrieval (2018)0.00
- Interactive Text-to-image Retrieval With Large Language Models: A Plug-and-play Approach (2024)10.24
- Recqr: Incorporating Conversational Query Rewriting To Improve Multimodal Image Retrieval (2026)0.00
- Photochat: A Human-human Dialogue Dataset With Photo Sharing Behavior For Joint Image-text Modeling (2021)9.92
- Enhancing Image Retrieval : A Comprehensive Study On Photo Search Using The CLIP Mode (2024)0.00
- Ask&confirm: Active Detail Enriching For Cross-modal Retrieval With Partial Query (2021)11.68
- Enhancing Interactive Image Retrieval With Query Rewriting Using Large Language Models And Vision Language Models (2024)8.82