Recurrence Meets Transformers For Universal Multimodal Retrieval

Abstract

With the rapid advancement of multimodal retrieval and its application in LLMs and multimodal LLMs, increasingly complex retrieval tasks have emerged. Existing methods predominantly rely on task-specific fine-tuning of vision-language models and are limited to single-modality queries or documents. In this paper, we propose ReT-2, a unified retrieval model that supports multimodal queries, composed of both images and text, and searches across multimodal document collections where text and images coexist. ReT-2 leverages multi-layer representations and a recurrent Transformer architecture with LSTM-inspired gating mechanisms to dynamically integrate information across layers and modalities, capturing fine-grained visual and textual details. We evaluate ReT-2 on the challenging M2KR and M-BEIR benchmarks across different retrieval configurations. Results demonstrate that ReT-2 consistently achieves state-of-the-art performance across diverse settings, while offering faster inference and r

Stats

• citations0
• S2 citations—
• github stars15
• HF likes0
• heat score2.41
• arxiv keycaffagni2025recurrence

Related papers

• Mm-embed: Universal Multimodal Retrieval With Multimodal Llms (2024)0.00
• RETLLM: Training And Data-free Mllms For Multimodal Information Retrieval (2026)1.57
• Indexing Multimodal Language Models For Large-scale Image Retrieval (2026)0.00
• Rematch: Boosting Representation Through Matching For Multimodal Retrieval (2025)0.00
• Lamra: Large Multimodal Model As Your Advanced Retrieval Assistant (2024)7.50
• TRACE: Task-adaptive Reasoning And Representation Learning For Universal Multimodal Retrieval (2026)0.00
• Tevatron 2.0: Unified Document Retrieval Toolkit Across Scale, Language, And Modality (2025)3.58
• Retrieval-augmented Multimodal Language Modeling (2022)0.00