Visual-rag: Benchmarking Text-to-image Retrieval Augmented Generation For Visual Knowledge Intensive Queries

Abstract

Retrieval-augmented generation (RAG) is a paradigm that augments large language models (LLMs) with external knowledge to tackle knowledge-intensive question answering. While several benchmarks evaluate Multimodal LLMs (MLLMs) under Multimodal RAG settings, they predominantly retrieve from textual corpora and do not explicitly assess how models exploit visual evidence during generation. Consequently, there still lacks benchmark that isolates and measures the contribution of retrieved images in RAG. We introduce Visual-RAG, a question-answering benchmark that targets visually grounded, knowledge-intensive questions. Unlike prior work, Visual-RAG requires text-to-image retrieval and the integration of retrieved clue images to extract visual evidence for answer generation. With Visual-RAG, we evaluate 5 open-source and 3 proprietary MLLMs, showcasing that images provide strong evidence in augmented generation. However, even state-of-the-art models struggle to efficiently extract and utiliz

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywu2025visual

Related papers

• Mrag-bench: Vision-centric Evaluation For Retrieval-augmented Multimodal Models (2024)0.00
• Cross-modal RAG: Sub-dimensional Text-to-image Retrieval-augmented Generation (2025)0.00
• M4-RAG: A Massive-scale Multilingual Multi-cultural Multimodal RAG (2025)2.00
• Murag: Multimodal Retrieval-augmented Generator For Open Question Answering Over Images And Text (2022)14.66
• Rag-check: Evaluating Multimodal Retrieval Augmented Generation Performance (2025)0.00
• Visrag 2.0: Evidence-guided Multi-image Reasoning In Visual Retrieval-augmented Generation (2025)0.00
• Enhancing Document VQA Models Via Retrieval-augmented Generation (2025)0.00
• Vdocrag: Retrieval-augmented Generation Over Visually-rich Documents (2025)6.34