Exploring Uncertainty Measures For Image-caption Embedding-and-retrieval Task

Abstract

With the wide development of black-box machine learning algorithms, particularly deep neural network (DNN), the practical demand for the reliability assessment is rapidly rising. On the basis of the concept that `Bayesian deep learning knows what it does not know,' the uncertainty of DNN outputs has been investigated as a reliability measure for the classification and regression tasks. However, in the image-caption retrieval task, well-known samples are not always easy-to-retrieve samples. This study investigates two aspects of image-caption embedding-and-retrieval systems. On one hand, we quantify feature uncertainty by considering image-caption embedding as a regression task, and use it for model averaging, which can improve the retrieval performance. On the other hand, we further quantify posterior uncertainty by considering the retrieval as a classification task, and use it as a reliability measure, which can greatly improve the retrieval performance by rejecting uncertain queries.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyhama2019exploring

Related papers

• Bayesian Triplet Loss: Uncertainty Quantification In Image Retrieval (2020)11.49
• Ranking-aware Uncertainty For Text-guided Image Retrieval (2023)0.00
• Exploring Uncertainty In Conditional Multi-modal Retrieval Systems (2019)0.00
• Unsupervised Data Uncertainty Learning In Visual Retrieval Systems (2019)0.00
• Reliability-aware Prediction Via Uncertainty Learning For Person Image Retrieval (2022)8.35
• Risk Controlled Image Retrieval (2023)0.00
• Uncertainty-based Cross-modal Retrieval With Probabilistic Representations (2022)0.00
• Introspective Deep Metric Learning For Image Retrieval (2022)3.50