Feature Re-learning With Data Augmentation For Video Relevance Prediction

Abstract

Predicting the relevance between two given videos with respect to their visual content is a key component for content-based video recommendation and retrieval. Thanks to the increasing availability of pre-trained image and video convolutional neural network models, deep visual features are widely used for video content representation. However, as how two videos are relevant is task-dependent, such off-the-shelf features are not always optimal for all tasks. Moreover, due to varied concerns including copyright, privacy and security, one might have access to only pre-computed video features rather than original videos. We propose in this paper feature re-learning for improving video relevance prediction, with no need of revisiting the original video content. In particular, re-learning is realized by projecting a given deep feature into a new space by an affine transformation. We optimize the re-learning process by a novel negative-enhanced triplet ranking loss. In order to generate more

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keydong2020feature

Related papers

• Learning Video Retrieval Models With Relevance-aware Online Mining (2022)6.07
• A Feature-space Multimodal Data Augmentation Technique For Text-video Retrieval (2022)12.43
• DREAM: Improving Video-text Retrieval Through Relevance-based Augmentation Using Large Foundation Models (2024)2.26
• Learning Test-time Augmentation For Content-based Image Retrieval (2020)5.24
• Relevance-based Margin For Contrastively-trained Video Retrieval Models (2022)7.74
• Video Retrieval Based On Deep Convolutional Neural Network (2017)9.03
• Dual Learning With Dynamic Knowledge Distillation And Soft Alignment For Partially Relevant Video Retrieval (2025)2.60
• Exploiting Local Indexing And Deep Feature Confidence Scores For Fast Image-to-video Search (2018)2.26