Not All Pairs Are Equal: Hierarchical Learning For Average-precision-oriented Video Retrieval

Abstract

The rapid growth of online video resources has significantly promoted the development of video retrieval methods. As a standard evaluation metric for video retrieval, Average Precision (AP) assesses the overall rankings of relevant videos at the top list, making the predicted scores a reliable reference for users. However, recent video retrieval methods utilize pair-wise losses that treat all sample pairs equally, leading to an evident gap between the training objective and evaluation metric. To effectively bridge this gap, in this work, we aim to address two primary challenges: a) The current similarity measure and AP-based loss are suboptimal for video retrieval; b) The noticeable noise from frame-to-frame matching introduces ambiguity in estimating the AP loss. In response to these challenges, we propose the Hierarchical learning framework for Average-Precision-oriented Video Retrieval (HAP-VR). For the former challenge, we develop the TopK-Chamfer Similarity and QuadLinear-AP loss

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keyliu2024not

Related papers

• Hierarchical Average Precision Training For Pertinent Image Retrieval (2022)9.10
• Hlformer: Enhancing Partially Relevant Video Retrieval With Hyperbolic Learning (2025)3.50
• Improving Video Retrieval By Adaptive Margin (2023)9.92
• Learning With Average Precision: Training Image Retrieval With A Listwise Loss (2019)18.86
• Delving Deeper: Hierarchical Visual Perception For Robust Video-text Retrieval (2026)1.24
• Towards Efficient And Robust Moment Retrieval System: A Unified Framework For Multi-granularity Models And Temporal Reranking (2025)2.26
• Prototypes Are Balanced Units For Efficient And Effective Partially Relevant Video Retrieval (2025)0.00
• UATVR: Uncertainty-adaptive Text-video Retrieval (2023)15.46