Normalized Contrastive Learning For Text-video Retrieval

Abstract

Cross-modal contrastive learning has led the recent advances in multimodal retrieval with its simplicity and effectiveness. In this work, however, we reveal that cross-modal contrastive learning suffers from incorrect normalization of the sum retrieval probabilities of each text or video instance. Specifically, we show that many test instances are either over- or under-represented during retrieval, significantly hurting the retrieval performance. To address this problem, we propose Normalized Contrastive Learning (NCL) which utilizes the Sinkhorn-Knopp algorithm to compute the instance-wise biases that properly normalize the sum retrieval probabilities of each instance so that every text and video instance is fairly represented during cross-modal retrieval. Empirical study shows that NCL brings consistent and significant gains in text-video retrieval on different model architectures, with new state-of-the-art multimodal retrieval metrics on the ActivityNet, MSVD, and MSR-VTT datasets w

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keypark2022normalized

Related papers

• Dual-modal Attention-enhanced Text-video Retrieval With Triplet Partial Margin Contrastive Learning (2023)8.82
• X-CLIP: End-to-end Multi-grained Contrastive Learning For Video-text Retrieval (2022)18.12
• Crossclr: Cross-modal Contrastive Learning For Multi-modal Video Representations (2021)15.59
• Generalized Contrastive Learning For Universal Multimodal Retrieval (2025)0.00
• Relevance-based Margin For Contrastively-trained Video Retrieval Models (2022)7.74
• TC-MGC: Text-conditioned Multi-grained Contrastive Learning For Text-video Retrieval (2025)6.93
• Video Corpus Moment Retrieval With Contrastive Learning (2021)14.35
• Rebalancing Contrastive Alignment With Bottlenecked Semantic Increments In Text-video Retrieval (2025)1.69