Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework

Abstract

We propose a self-supervised method to learn feature representations from videos. A standard approach in traditional self-supervised methods uses positive-negative data pairs to train with contrastive learning strategy. In such a case, different modalities of the same video are treated as positives and video clips from a different video are treated as negatives. Because the spatio-temporal information is important for video representation, we extend the negative samples by introducing intra-negative samples, which are transformed from the same anchor video by breaking temporal relations in video clips. With the proposed Inter-Intra Contrastive (IIC) framework, we can train spatio-temporal convolutional networks to learn video representations. There are many flexible options in our IIC framework and we conduct experiments by using several different configurations. Evaluations are conducted on video retrieval and video recognition tasks using the learned video representation. Our propose

Stats

• citations84
• S2 citations—
• github stars112
• HF likes0
• heat score18.58
• arxiv keytao2020self

Related papers

• Nearest-neighbor Inter-intra Contrastive Learning From Unlabeled Videos (2023)0.00
• TCLR: Temporal Contrastive Learning For Video Representation (2021)15.78
• Cycle-contrast For Self-supervised Video Representation Learning (2020)0.00
• Self-supervised Video Representation Learning With Cross-stream Prototypical Contrasting (2021)8.82
• Self-supervised Video Representation Learning With Meta-contrastive Network (2021)11.85
• Multimodal Clustering Networks For Self-supervised Learning From Unlabeled Videos (2021)13.28
• Temporal Context Aggregation For Video Retrieval With Contrastive Learning (2020)13.23
• Latent-inr: A Flexible Framework For Implicit Representations Of Videos With Discriminative Semantics (2024)5.84