Mixed-modality Representation Learning And Pre-training For Joint Table-and-text Retrieval In Openqa

Abstract

Retrieving evidences from tabular and textual resources is essential for open-domain question answering (OpenQA), which provides more comprehensive information. However, training an effective dense table-text retriever is difficult due to the challenges of table-text discrepancy and data sparsity problem. To address the above challenges, we introduce an optimized OpenQA Table-Text Retriever (OTTeR) to jointly retrieve tabular and textual evidences. Firstly, we propose to enhance mixed-modality representation learning via two mechanisms: modality-enhanced representation and mixed-modality negative sampling strategy. Secondly, to alleviate data sparsity problem and enhance the general retrieval ability, we conduct retrieval-centric mixed-modality synthetic pre-training. Experimental results demonstrate that OTTeR substantially improves the performance of table-and-text retrieval on the OTT-QA dataset. Comprehensive analyses examine the effectiveness of all the proposed mechanisms. Beside

Stats

• citations6
• S2 citations—
• github stars21
• HF likes0
• heat score9.02
• arxiv keyhuang2022mixed

Related papers

• Multi-modal Retrieval Of Tables And Texts Using Tri-encoder Models (2021)6.34
• CGPT: Cluster-guided Partial Tables With Llm-generated Supervision For Table Retrieval (2026)1.57
• LITTA: Late-interaction And Test-time Alignment For Visually-grounded Multimodal Retrieval (2026)0.00
• End-to-end Knowledge Retrieval With Multi-modal Queries (2023)8.35
• An Interactive Multi-modal Query Answering System With Retrieval-augmented Large Language Models (2024)5.84
• TRACE: Task-adaptive Reasoning And Representation Learning For Universal Multimodal Retrieval (2026)0.00
• Enhancing Question Answering Precision With Optimized Vector Retrieval And Instructions (2024)0.00
• Mllms-augmented Visual-language Representation Learning (2023)0.00