Ask&confirm: Active Detail Enriching For Cross-modal Retrieval With Partial Query
2021 · Guanyu Cai, Jun Zhang, Xinyang Jiang, et al.
Abstract
Text-based image retrieval has seen considerable progress in recent years. However, the performance of existing methods suffers in real life since the user is likely to provide an incomplete description of an image, which often leads to results filled with false positives that fit the incomplete description. In this work, we introduce the partial-query problem and extensively analyze its influence on text-based image retrieval. Previous interactive methods tackle the problem by passively receiving users' feedback to supplement the incomplete query iteratively, which is time-consuming and requires heavy user effort. Instead, we propose a novel retrieval framework that conducts the interactive process in an Ask-and-Confirm fashion, where AI actively searches for discriminative details missing in the current query, and users only need to confirm AI's proposal. Specifically, we propose an object-based interaction to make the interactive retrieval more user-friendly and present a reinforcem
Authors
(none)
Tags
Stats
Related papers
- Dialog-based Interactive Image Retrieval (2018)0.00
- Image Retrieval With Mixed Initiative And Multimodal Feedback (2018)8.09
- Telling The What While Pointing To The Where: Multimodal Queries For Image Retrieval (2021)10.07
- Maybe You Are Looking For Croqs: Cross-modal Query Suggestion For Text-to-image Retrieval (2024)0.00
- Attention Grounded Enhancement For Visual Document Retrieval (2025)0.00
- Simple Baselines For Interactive Video Retrieval With Questions And Answers (2023)7.16
- Extending Cross-modal Retrieval With Interactive Learning To Improve Image Retrieval Performance In Forensics (2023)0.00
- End-to-end Knowledge Retrieval With Multi-modal Queries (2023)8.35