TC-MGC: Text-conditioned Multi-grained Contrastive Learning For Text-video Retrieval

Abstract

Motivated by the success of coarse-grained or fine-grained contrast in text-video retrieval, there emerge multi-grained contrastive learning methods which focus on the integration of contrasts with different granularity. However, due to the wider semantic range of videos, the text-agnostic video representations might encode misleading information not described in texts, thus impeding the model from capturing precise cross-modal semantic correspondence. To this end, we propose a Text-Conditioned Multi-Grained Contrast framework, dubbed TC-MGC. Specifically, our model employs a language-video attention block to generate aggregated frame and video representations conditioned on the word's and text's attention weights over frames. To filter unnecessary similarity interactions and decrease trainable parameters in the Interactive Similarity Aggregation (ISA) module, we design a Similarity Reorganization (SR) module to identify attentive similarities and reorganize cross-modal similarity vect

Stats

• citations4
• S2 citations—
• github stars6
• HF likes0
• heat score6.93
• arxiv keyjing2025tc

Related papers

• X-CLIP: End-to-end Multi-grained Contrastive Learning For Video-text Retrieval (2022)18.12
• Temporal Context Aggregation For Video Retrieval With Contrastive Learning (2020)13.23
• Dual-modal Attention-enhanced Text-video Retrieval With Triplet Partial Margin Contrastive Learning (2023)8.82
• Normalized Contrastive Learning For Text-video Retrieval (2022)6.77
• Tencent Text-video Retrieval: Hierarchical Cross-modal Interactions With Multi-level Representations (2022)7.81
• Video Corpus Moment Retrieval With Contrastive Learning (2021)14.35
• MDMMT-2: Multidomain Multimodal Transformer For Video Retrieval, One More Step Towards Generalization (2022)0.00
• Towards Fast Adaptation Of Pretrained Contrastive Models For Multi-channel Video-language Retrieval (2022)7.50