NAVERO: Unlocking Fine-grained Semantics For Video-language Compositionality

Abstract

We study the capability of Video-Language (VidL) models in understanding compositions between objects, attributes, actions and their relations. Composition understanding becomes particularly challenging for video data since the compositional relations rapidly change over time in videos. We first build a benchmark named AARO to evaluate composition understanding related to actions on top of spatial concepts. The benchmark is constructed by generating negative texts with incorrect action descriptions for a given video and the model is expected to pair a positive text with its corresponding video. Furthermore, we propose a training method called NAVERO which utilizes video-text data augmented with negative texts to enhance composition understanding. We also develop a negative-augmented visual-language matching loss which is used explicitly to benefit from the generated negative text. We compare NAVERO with other state-of-the-art methods in terms of compositional understanding as well as v

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytao2024navero

Related papers

• ICSVR: Investigating Compositional And Syntactic Understanding In Video Retrieval Models (2023)8.92
• RAVU: Retrieval Augmented Video Understanding With Compositional Reasoning Over Graph (2025)0.00
• Contrasting Intra-modal And Ranking Cross-modal Hard Negatives To Enhance Visio-linguistic Compositional Understanding (2023)12.11
• Video-adverb Retrieval With Compositional Adverb-action Embeddings (2023)0.00
• Narrating The Video: Boosting Text-video Retrieval Via Comprehensive Utilization Of Frame-level Captions (2025)6.77
• Linear Spaces Of Meanings: Compositional Structures In Vision-language Models (2023)9.41
• Lazyvlm: Neuro-symbolic Approach To Video Analytics (2025)0.00
• Learning Visual Composition Through Improved Semantic Guidance (2024)0.00