Relevance-based Margin For Contrastively-trained Video Retrieval Models

Abstract

Video retrieval using natural language queries has attracted increasing interest due to its relevance in real-world applications, from intelligent access in private media galleries to web-scale video search. Learning the cross-similarity of video and text in a joint embedding space is the dominant approach. To do so, a contrastive loss is usually employed because it organizes the embedding space by putting similar items close and dissimilar items far. This framework leads to competitive recall rates, as they solely focus on the rank of the groundtruth items. Yet, assessing the quality of the ranking list is of utmost importance when considering intelligent retrieval systems, since multiple items may share similar semantics, hence a high relevance. Moreover, the aforementioned framework uses a fixed margin to separate similar and dissimilar items, treating all non-groundtruth items as equally irrelevant. In this paper we propose to use a variable margin: we argue that varying the margin

Stats

• citations6
• S2 citations—
• github stars4
• HF likes0
• heat score7.74
• arxiv keyfalcon2022relevance

Related papers

• Improving Video Retrieval By Adaptive Margin (2023)9.92
• Dual-modal Attention-enhanced Text-video Retrieval With Triplet Partial Margin Contrastive Learning (2023)8.82
• Normalized Contrastive Learning For Text-video Retrieval (2022)6.77
• X-CLIP: End-to-end Multi-grained Contrastive Learning For Video-text Retrieval (2022)18.12
• Modality-balanced Embedding For Video Retrieval (2022)7.16
• Rebalancing Contrastive Alignment With Bottlenecked Semantic Increments In Text-video Retrieval (2025)1.69
• Learning Video Retrieval Models With Relevance-aware Online Mining (2022)6.07
• Towards Fast Adaptation Of Pretrained Contrastive Models For Multi-channel Video-language Retrieval (2022)7.50