Give Me A Hint! Navigating Image Databases Using Human-in-the-loop Feedback

Abstract

In this paper, we introduce an attribute-based interactive image search which can leverage human-in-the-loop feedback to iteratively refine image search results. We study active image search where human feedback is solicited exclusively in visual form, without using relative attribute annotations used by prior work which are not typically found in many datasets. In order to optimize the image selection strategy, a deep reinforcement model is trained to learn what images are informative rather than rely on hand-crafted measures typically leveraged in prior work. Additionally, we extend the recently introduced Conditional Similarity Network to incorporate global similarity in training visual embeddings, which results in more natural transitions as the user explores the learned similarity embeddings. Our experiments demonstrate the effectiveness of our approach, producing compelling results on both active image search and image attribute representation tasks.

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keyplummer2018give

Related papers

• Image Retrieval With Mixed Initiative And Multimodal Feedback (2018)8.09
• Dialog-based Interactive Image Retrieval (2018)0.00
• Attribute-aware Deep Hashing With Self-consistency For Large-scale Fine-grained Image Retrieval (2023)11.76
• Seesaw: Interactive Ad-hoc Search Over Image Databases (2022)5.24
• Fast Interactive Image Retrieval Using Large-scale Unlabeled Data (2018)0.00
• Information-theoretic Active Learning For Content-based Image Retrieval (2018)5.24
• Connecting Images Through Time And Sources: Introducing Low-data, Heterogeneous Instance Retrieval (2021)0.00
• Ask&confirm: Active Detail Enriching For Cross-modal Retrieval With Partial Query (2021)11.68