Differentiable Resolution Compression And Alignment For Efficient Video Classification And Retrieval

Abstract

Optimizing video inference efficiency has become increasingly important with the growing demand for video analysis in various fields. Some existing methods achieve high efficiency by explicit discard of spatial or temporal information, which poses challenges in fast-changing and fine-grained scenarios. To address these issues, we propose an efficient video representation network with Differentiable Resolution Compression and Alignment mechanism, which compresses non-essential information in the early stage of the network to reduce computational costs while maintaining consistent temporal correlations. Specifically, we leverage a Differentiable Context-aware Compression Module to encode the saliency and non-saliency frame features, refining and updating the features into a high-low resolution video sequence. To process the new sequence, we introduce a new Resolution-Align Transformer Layer to capture global temporal correlations among frame features with different resolutions, while red

Stats

• citations2
• S2 citations—
• github stars6
• HF likes0
• heat score5.27
• arxiv keydeng2023differentiable

Related papers

• Exploiting Local Indexing And Deep Feature Confidence Scores For Fast Image-to-video Search (2018)2.26
• Latent-inr: A Flexible Framework For Implicit Representations Of Videos With Discriminative Semantics (2024)5.84
• Hybrid Contrastive Quantization For Efficient Cross-view Video Retrieval (2022)9.28
• Prompt Switch: Efficient CLIP Adaptation For Text-video Retrieval (2023)11.93
• Discovla: Discrepancy Reduction In Vision, Language, And Alignment For Parameter-efficient Video-text Retrieval (2025)6.30
• CHAIN: Exploring Global-local Spatio-temporal Information For Improved Self-supervised Video Hashing (2023)8.60
• Encode The Unseen: Predictive Video Hashing For Scalable Mid-stream Retrieval (2020)3.58
• Towards Efficient And Robust Moment Retrieval System: A Unified Framework For Multi-granularity Models And Temporal Reranking (2025)2.26