Transfer Learning Approaches For Building Cross-language Dense Retrieval Models
2022 · Suraj Nair, Eugene Yang, Dawn Lawrie, et al.
Abstract
The advent of transformer-based models such as BERT has led to the rise of neural ranking models. These models have improved the effectiveness of retrieval systems well beyond that of lexical term matching models such as BM25. While monolingual retrieval tasks have benefited from large-scale training collections such as MS MARCO and advances in neural architectures, cross-language retrieval tasks have fallen behind these advancements. This paper introduces ColBERT-X, a generalization of the ColBERT multi-representation dense retrieval model that uses the XLM-RoBERTa (XLM-R) encoder to support cross-language information retrieval (CLIR). ColBERT-X can be trained in two ways. In zero-shot training, the system is trained on the English MS MARCO collection, relying on the XLM-R encoder for cross-language mappings. In translate-train, the system is trained on the MS MARCO English queries coupled with machine translations of the associated MS MARCO passages. Results on ad hoc document rankin
Authors
(none)
Tags
Stats
Related papers
- Colbert-xm: A Modular Multi-vector Representation Model For Zero-shot Multilingual Information Retrieval (2024)0.00
- Boosting Zero-shot Cross-lingual Retrieval By Training On Artificially Code-switched Data (2023)4.52
- Evaluating Multilingual Text Encoders For Unsupervised Cross-lingual Retrieval (2021)7.50
- Translate-distill: Learning Cross-language Dense Retrieval By Translation And Distillation (2024)8.60
- What Drives Cross-lingual Ranking? Retrieval Approaches With Multilingual Language Models (2025)0.00
- Improving Cross-lingual Information Retrieval On Low-resource Languages Via Optimal Transport Distillation (2023)10.07
- Jina-colbert-v2: A General-purpose Multilingual Late Interaction Retriever (2024)5.24
- CL2CM: Improving Cross-lingual Cross-modal Retrieval Via Cross-lingual Knowledge Transfer (2023)8.60