Dual-modal Attention-enhanced Text-video Retrieval With Triplet Partial Margin Contrastive Learning

Abstract

In recent years, the explosion of web videos makes text-video retrieval increasingly essential and popular for video filtering, recommendation, and search. Text-video retrieval aims to rank relevant text/video higher than irrelevant ones. The core of this task is to precisely measure the cross-modal similarity between texts and videos. Recently, contrastive learning methods have shown promising results for text-video retrieval, most of which focus on the construction of positive and negative pairs to learn text and video representations. Nevertheless, they do not pay enough attention to hard negative pairs and lack the ability to model different levels of semantic similarity. To address these two issues, this paper improves contrastive learning using two novel techniques. First, to exploit hard examples for robust discriminative power, we propose a novel Dual-Modal Attention-Enhanced Module (DMAE) to mine hard negative pairs from textual and visual clues. By further introducing a Negat

Stats

• citations14
• S2 citations—
• github stars0
• HF likes0
• heat score8.82
• arxiv keyjiang2023dual

Related papers

• Normalized Contrastive Learning For Text-video Retrieval (2022)6.77
• Memory Enhanced Embedding Learning For Cross-modal Video-text Retrieval (2021)0.00
• Relevance-based Margin For Contrastively-trained Video Retrieval Models (2022)7.74
• X-CLIP: End-to-end Multi-grained Contrastive Learning For Video-text Retrieval (2022)18.12
• TC-MGC: Text-conditioned Multi-grained Contrastive Learning For Text-video Retrieval (2025)6.93
• Hit: Hierarchical Transformer With Momentum Contrast For Video-text Retrieval (2021)15.98
• Improving Video Retrieval By Adaptive Margin (2023)9.92
• Improving Video-text Retrieval By Multi-stream Corpus Alignment And Dual Softmax Loss (2021)0.00