Ranking-aware Uncertainty For Text-guided Image Retrieval

Abstract

Text-guided image retrieval is to incorporate conditional text to better capture users' intent. Traditionally, the existing methods focus on minimizing the embedding distances between the source inputs and the targeted image, using the provided triplets \(\langle\)source image, source text, target image\(\rangle\). However, such triplet optimization may limit the learned retrieval model to capture more detailed ranking information, e.g., the triplets are one-to-one correspondences and they fail to account for many-to-many correspondences arising from semantic diversity in feedback languages and images. To capture more ranking information, we propose a novel ranking-aware uncertainty approach to model many-to-many correspondences by only using the provided triplets. We introduce uncertainty learning to learn the stochastic ranking list of features. Specifically, our approach mainly comprises three components: (1) In-sample uncertainty, which aims to capture semantic diversity using a Ga

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2023ranking

Related papers

• Composed Image Retrieval With Text Feedback Via Multi-grained Uncertainty Regularization (2022)0.00
• Bayesian Triplet Loss: Uncertainty Quantification In Image Retrieval (2020)11.49
• Unsupervised Data Uncertainty Learning In Visual Retrieval Systems (2019)0.00
• Uncertainty-based Cross-modal Retrieval With Probabilistic Representations (2022)0.00
• Heterogeneous Uncertainty-guided Composed Image Retrieval With Fine-grained Probabilistic Learning (2026)0.00
• UATVR: Uncertainty-adaptive Text-video Retrieval (2023)15.46
• Exploring Uncertainty Measures For Image-caption Embedding-and-retrieval Task (2019)2.26
• Reliability-aware Prediction Via Uncertainty Learning For Person Image Retrieval (2022)8.35