Not All Relevance Scores Are Equal: Efficient Uncertainty And Calibration Modeling For Deep Retrieval Models
2021 · Daniel Cohen, Bhaskar Mitra, Oleg Lesota, et al.
Abstract
In any ranking system, the retrieval model outputs a single score for a document based on its belief on how relevant it is to a given search query. While retrieval models have continued to improve with the introduction of increasingly complex architectures, few works have investigated a retrieval model's belief in the score beyond the scope of a single value. We argue that capturing the model's uncertainty with respect to its own scoring of a document is a critical aspect of retrieval that allows for greater use of current models across new document distributions, collections, or even improving effectiveness for down-stream tasks. In this paper, we address this problem via an efficient Bayesian framework for retrieval models which captures the model's belief in the relevance score through a stochastic process while adding only negligible computational overhead. We evaluate this belief via a ranking based calibration metric showing that our approximate Bayesian framework significantly i
Authors
(none)
Tags
Stats
Related papers
- On The Calibration And Uncertainty Of Neural Learning To Rank Models (2021)0.00
- Reliable Evaluation Protocol For Low-precision Retrieval (2025)0.00
- Enhancing The Ranking Context Of Dense Retrieval Methods Through Reciprocal Nearest Neighbors (2023)4.52
- The Overlooked Role Of Graded Relevance Thresholds In Multilingual Dense Retrieval (2026)0.00
- Exploring Uncertainty Measures For Image-caption Embedding-and-retrieval Task (2019)2.26
- A Deep Look Into Neural Ranking Models For Information Retrieval (2019)17.73
- Modeling Diverse Relevance Patterns In Ad-hoc Retrieval (2018)12.25
- Ranking-aware Uncertainty For Text-guided Image Retrieval (2023)0.00