Transvcl: Attention-enhanced Video Copy Localization Network With Flexible Supervision

Abstract

Video copy localization aims to precisely localize all the copied segments within a pair of untrimmed videos in video retrieval applications. Previous methods typically start from frame-to-frame similarity matrix generated by cosine similarity between frame-level features of the input video pair, and then detect and refine the boundaries of copied segments on similarity matrix under temporal constraints. In this paper, we propose TransVCL: an attention-enhanced video copy localization network, which is optimized directly from initial frame-level features and trained end-to-end with three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for similarity matrix generation, and a temporal alignment module for copied segments localization. In contrast to previous methods demanding the handcrafted similarity matrix, TransVCL incorporates long-range temporal information between feature sequence pair using self- and cross- attention layers. With

Stats

• citations20
• S2 citations—
• github stars59
• HF likes0
• heat score13.47
• arxiv keyhe2022transvcl

Related papers

• Locvtp: Video-text Pre-training For Temporal Localization (2022)11.39
• Self-supervised Video Retrieval Transformer Network (2021)0.00
• Vlanet: Video-language Alignment Network For Weakly-supervised Video Moment Retrieval (2020)13.28
• Mv-adapter: Multimodal Video Transfer Learning For Video Text Retrieval (2023)9.76
• Clip2video: Mastering Video-text Retrieval Via Image CLIP (2021)0.00
• CLIP2TV: Align, Match And Distill For Video-text Retrieval (2021)0.00
• Improving Video-text Retrieval By Multi-stream Corpus Alignment And Dual Softmax Loss (2021)0.00
• Centerclip: Token Clustering For Efficient Text-video Retrieval (2022)15.54