Attention Grounded Enhancement For Visual Document Retrieval

Abstract

Visual document retrieval requires understanding heterogeneous and multi-modal content to satisfy information needs. Recent advances use screenshot-based document encoding with fine-grained late interaction, significantly improving retrieval performance. However, retrievers are still trained with coarse global relevance labels, without revealing which regions support the match. As a result, retrievers tend to rely on surface-level cues and struggle to capture implicit semantic connections, hindering their ability to handle non-extractive queries. To alleviate this problem, we propose a \textbf\{A\}ttention-\textbf\{G\}rounded \textbf\{RE\}triever \textbf\{E\}nhancement (AGREE) framework. AGREE leverages cross-modal attention from multimodal large language models as proxy local supervision to guide the identification of relevant document regions. During training, AGREE combines local signals with the global signals to jointly optimize the retriever, enabling it to learn not only whether

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycui2025attention

Related papers

• Realign: Optimizing The Visual Document Retriever With Reasoning-guided Fine-grained Alignment (2026)2.20
• Regionrag: Region-level Retrieval-augmented Generation For Visual Document Understanding (2025)0.00
• Modernvbert: Towards Smaller Visual Document Retrievers (2025)0.00
• Spatially-grounded Document Retrieval Via Patch-to-region Relevance Propagation (2025)3.86
• Document Optimization For Black-box Retrieval Via Reinforcement Learning (2026)0.00
• Gear: Generation Augmented Retrieval (2025)2.79
• Combating Visual Neglect And Semantic Drift In Large Multimodal Models For Enhanced Cross-modal Retrieval (2026)0.00
• Evo-retriever: Llm-guided Curriculum Evolution With Viewpoint-pathway Collaboration For Multimodal Document Retrieval (2026)0.00