Neuclirbench: A Modern Evaluation Collection For Monolingual, Cross-language, And Multilingual Information Retrieval

Abstract

To measure advances in retrieval, test collections with relevance judgments that can faithfully distinguish systems are required. This paper presents NeuCLIRBench, an evaluation collection for cross-language and multilingual retrieval. The collection consists of documents written natively in Chinese, Persian, and Russian, as well as those same documents machine translated into English. The collection supports several retrieval scenarios including: monolingual retrieval in English, Chinese, Persian, or Russian; cross-language retrieval with English as the query language and one of the other three languages as the document language; and multilingual retrieval, again with English as the query language and relevant documents in all three languages. NeuCLIRBench combines the TREC NeuCLIR track topics of 2022, 2023, and 2024. The 250,128 judgments across approximately 150 queries for the monolingual and cross-language tasks and 100 queries for multilingual retrieval provide strong statistica

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylawrie2025neuclirbench

Related papers

• Clirudit: Cross-lingual Information Retrieval Of Scientific Documents (2025)0.00
• Mfollowir: A Multilingual Benchmark For Instruction Following In Retrieval (2025)0.00
• What Drives Cross-lingual Ranking? Retrieval Approaches With Multilingual Language Models (2025)0.00
• Bridging Language Gaps: Advances In Cross-lingual Information Retrieval With Multilingual Llms (2025)0.00
• Visr-bench: An Empirical Study On Visual Retrieval-augmented Generation For Multilingual Long Document Understanding (2025)0.00
• On Cross-lingual Retrieval With Multilingual Text Encoders (2021)10.35
• Evaluating Multilingual Text Encoders For Unsupervised Cross-lingual Retrieval (2021)7.50
• Translate-distill: Learning Cross-language Dense Retrieval By Translation And Distillation (2024)8.60