Self-supervised Video Similarity Learning

Abstract

We introduce S\(^2\)VS, a video similarity learning approach with self-supervision. Self-Supervised Learning (SSL) is typically used to train deep models on a proxy task so as to have strong transferability on target tasks after fine-tuning. Here, in contrast to prior work, SSL is used to perform video similarity learning and address multiple retrieval and detection tasks at once with no use of labeled data. This is achieved by learning via instance-discrimination with task-tailored augmentations and the widely used InfoNCE loss together with an additional loss operating jointly on self-similarity and hard-negative similarity. We benchmark our method on tasks where video relevance is defined with varying granularity, ranging from video copies to videos depicting the same incident or event. We learn a single universal model that achieves state-of-the-art performance on all tasks, surpassing previously proposed methods that use labeled data. The code and pretrained models are publicly av

Stats

• citations19
• S2 citations—
• github stars43
• HF likes0
• heat score13.04
• arxiv keykordopatiszilos2023self

Related papers

• 3D-CSL: Self-supervised 3D Context Similarity Learning For Near-duplicate Video Retrieval (2022)6.34
• Visil: Fine-grained Spatio-temporal Video Similarity Learning (2019)13.70
• Visual Representation Learning With Self-supervised Attention For Low-label High-data Regime (2022)5.49
• Self-supervised Video Retrieval Transformer Network (2021)0.00
• Convis-bench: Estimating Video Similarity Through Semantic Concepts (2025)0.00
• Loss Switching Fusion With Similarity Search For Video Classification (2019)10.21
• Nearest-neighbor Inter-intra Contrastive Learning From Unlabeled Videos (2023)0.00
• Self-supervising Fine-grained Region Similarities For Large-scale Image Localization (2020)15.85