Sovabench: A Vehicle Surveillance Action Retrieval Benchmark For Multimodal Large Language Models

Abstract

Automatic identification of events and recurrent behavior analysis are critical for video surveillance. However, most existing content-based video retrieval benchmarks focus on scene-level similarity and do not evaluate the action discrimination required in surveillance. To address this gap, we introduce SOVABench (Surveillance Opposite Vehicle Actions Benchmark), a real-world retrieval benchmark built from surveillance footage and centered on vehicle-related actions. SOVABench defines two evaluation protocols (inter-pair and intra-pair) to assess cross-action discrimination and temporal direction understanding. Although action distinctions are generally intuitive for human observers, our experiments show that they remain challenging for state-of-the-art vision and multimodal models. Leveraging the visual reasoning and instruction-following capabilities of Multimodal Large Language Models (MLLMs), we present a training-free framework for producing interpretable embeddings from MLLM-g

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrabasseda2026sovabench

Related papers

• Momentseeker: A Task-oriented Benchmark For Long-video Moment Retrieval (2025)0.00
• Lovr: A Benchmark For Long Video Retrieval In Multimodal Contexts (2025)0.00
• Multivent 2.0: A Massive Multilingual Benchmark For Event-centric Video Retrieval (2024)3.58
• Verve: Versatile Retrieval For Videos Via Unified Embeddings (2026)0.00
• Vision-deepresearch Benchmark: Rethinking Visual And Textual Search For Multimodal Large Language Models (2026)7.27
• MMMORRF: Multimodal Multilingual Modularized Reciprocal Rank Fusion (2025)2.26
• MUVR: A Multi-modal Untrimmed Video Retrieval Benchmark With Multi-level Visual Correspondence (2025)1.40
• Indexing Multimodal Language Models For Large-scale Image Retrieval (2026)0.00