Overview Of The TREC 2021 Deep Learning Track

Abstract

This is the third year of the TREC Deep Learning track. As in previous years, we leverage the MS MARCO datasets that made hundreds of thousands of human annotated training labels available for both passage and document ranking tasks. In addition, this year we refreshed both the document and the passage collections which also led to a nearly four times increase in the document collection size and nearly \(16\) times increase in the size of the passage collection. Deep neural ranking models that employ large scale pretraininig continued to outperform traditional retrieval methods this year. We also found that single stage retrieval can achieve good performance on both tasks although they still do not perform at par with multistage retrieval pipelines. Finally, the increase in the collection size and the general data refresh raised some questions about completeness of NIST judgments and the quality of the training labels that were mapped to the new collections from the old ones which we d

Stats

• citations27
• S2 citations—
• github stars0
• HF likes0
• heat score10.85
• arxiv keycraswell2025overview

Related papers

• Overview Of The TREC 2023 Product Product Search Track (2023)0.00
• Naver Labs Europe (SPLADE) @ TREC Deep Learning 2022 (2023)0.00
• Team IELAB At TREC Clinical Trial Track 2023: Enhancing Clinical Trial Retrieval With Neural Rankers And Large Language Models (2024)0.00
• How Different Are Pre-trained Transformers For Text Ranking? (2022)7.81
• PASH At TREC 2021 Deep Learning Track: Generative Enhanced Model For Multi-stage Ranking (2022)0.00
• The Tale Of Two MS MARCO -- And Their Unfair Comparisons (2023)6.34
• Generative Relevance Feedback And Convergence Of Adaptive Re-ranking: University Of Glasgow Terrier Team At TREC DL 2023 (2024)0.00
• Blending Learning To Rank And Dense Representations For Efficient And Effective Cascades (2025)0.00