Disentangled Representation Learning For Text-video Retrieval

Abstract

Cross-modality interaction is a critical component in Text-Video Retrieval (TVR), yet there has been little examination of how different influencing factors for computing interaction affect performance. This paper first studies the interaction paradigm in depth, where we find that its computation can be split into two terms, the interaction contents at different granularity and the matching function to distinguish pairs with the same semantics. We also observe that the single-vector representation and implicit intensive function substantially hinder the optimization. Based on these findings, we propose a disentangled framework to capture a sequential and hierarchical representation. Firstly, considering the natural sequential structure in both text and video inputs, a Weighted Token-wise Interaction (WTI) module is performed to decouple the content and adaptively exploit the pair-wise correlations. This interaction can form a better disentangled manifold for sequential inputs. Secondly

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2022disentangled

Related papers

• Video-colbert: Contextualized Late Interaction For Text-to-video Retrieval (2025)5.24
• Tencent Text-video Retrieval: Hierarchical Cross-modal Interactions With Multi-level Representations (2022)7.81
• UATVR: Uncertainty-adaptive Text-video Retrieval (2023)15.46
• Dual Encoding For Video Retrieval By Text (2020)16.05
• Bridging Information Asymmetry In Text-video Retrieval: A Data-centric Approach (2024)0.00
• Prota: Probabilistic Token Aggregation For Text-video Retrieval (2024)4.52
• T2vparser: Adaptive Decomposition Tokens For Partial Alignment In Text To Video Retrieval (2025)0.95
• Tree-augmented Cross-modal Encoding For Complex-query Video Retrieval (2020)15.57