Performance Evaluation In Multimedia Retrieval

Abstract

Performance evaluation in multimedia retrieval, as in the information retrieval domain at large, relies heavily on retrieval experiments, employing a broad range of techniques and metrics. These can involve human-in-the-loop and machine-only settings for the retrieval process itself and the subsequent verification of results. Such experiments can be elaborate and use-case-specific, which can make them difficult to compare or replicate. In this paper, we present a formal model to express all relevant aspects of such retrieval experiments, as well as a flexible open-source evaluation infrastructure that implements the model. These contributions intend to make a step towards lowering the hurdles for conducting retrieval experiments and improving their reproducibility.

Stats

• citations14
• S2 citations—
• github stars0
• HF likes0
• heat score8.82
• arxiv keysauter2024performance

Related papers

• An Intrinsic Framework Of Information Retrieval Evaluation Measures (2023)2.26
• Where Does The Performance Improvement Come From? -- A Reproducibility Concern About Image-text Retrieval (2022)3.36
• Multimodal Lengthy Videos Retrieval Framework And Evaluation Metric (2025)0.00
• Evaluating Interpolation And Extrapolation Performance Of Neural Retrieval Models (2022)6.77
• Cross-media Similarity Evaluation For Web Image Retrieval In The Wild (2017)9.59
• Machine Learning Methods For Multimedia Information Retrieval (2017)0.00
• Towards An All-purpose Content-based Multimedia Information Retrieval System (2019)0.00
• Retrieval-enhanced Machine Learning (2022)11.93