Mr\(^2\)-bench: Going Beyond Matching To Reasoning In Multimodal Retrieval

Abstract

Multimodal retrieval is becoming a crucial component of modern AI applications, yet its evaluation lags behind the demands of more realistic and challenging scenarios. Existing benchmarks primarily probe surface-level semantic correspondence (e.g., object-text matching) while failing to assess the deeper reasoning required to capture complex relationships between visual and textual information. To address this gap, we introduce MR\(^2\)-Bench, a reasoning-intensive benchmark for multimodal retrieval. MR\(^2\)-Bench presents the following critical values: 1) all tasks are reasoning-driven, going beyond shallow matching to effectively assess models' capacity for logical, spatial, and causal inference; 2) it features diverse multimodal data, such as natural images, diagrams, and visual puzzles, enabling comprehensive evaluation across content types; 3) it supports complex queries and documents containing multiple images and covers diverse retrieval scenarios, more accurately reflecting re

Stats

• citations0
• S2 citations—
• github stars7
• HF likes0
• heat score1.81
• arxiv keyzhou2025mr

Related papers

• MRMR: A Realistic And Expert-level Multidisciplinary Benchmark For Reasoning-intensive Multimodal Retrieval (2025)0.00
• MM-BRIGHT: A Multi-task Multimodal Benchmark For Reasoning-intensive Retrieval (2026)2.60
• MARVEL: Multimodal Adaptive Reasoning-intensive Expand-rerank And Retrieval (2026)0.00
• Multihaystack: Benchmarking Multimodal Retrieval And Reasoning Over 40K Images, Videos, And Documents (2026)0.00
• Mrag-bench: Vision-centric Evaluation For Retrieval-augmented Multimodal Models (2024)0.00
• Reasoning-augmented Representations For Multimodal Retrieval (2026)0.00
• Reason To Contrast: A Cascaded Multimodal Retrieval Framework (2025)0.00
• Beyond Global Similarity: Towards Fine-grained, Multi-condition Multimodal Retrieval (2026)2.20