Symmetric Multi-similarity Loss For EPIC-KITCHENS-100 Multi-instance Retrieval Challenge 2024

Abstract

In this report, we present our champion solution for EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge in CVPR 2024. Essentially, this challenge differs from traditional visual-text retrieval tasks by providing a correlation matrix that acts as a set of soft labels for video-text clip combinations. However, existing loss functions have not fully exploited this information. Motivated by this, we propose a novel loss function, Symmetric Multi-Similarity Loss, which offers a more precise learning objective. Together with tricks and ensemble learning, the model achieves 63.76% average mAP and 74.25% average nDCG on the public leaderboard, demonstrating the effectiveness of our approach. Our code will be released at: https://github.com/xqwang14/SMS-Loss/tree/main

Stats

• citations0
• S2 citations—
• github stars3
• HF likes0
• heat score1.20
• arxiv keywang2024symmetric

Code

• xqwang14/SMS-Loss—

Related papers

• Exploiting Semantic Role Contextualized Video Features For Multi-instance Text-video Retrieval EPIC-KITCHENS-100 Multi-instance Retrieval Challenge 2022 (2022)0.00
• Uniud Submission To The Epic-kitchens-100 Multi-instance Retrieval Challenge 2023 (2023)0.00
• Contextrefine-clip For EPIC-KITCHENS-100 Multi-instance Retrieval Challenge 2025 (2025)0.95
• Uniud-fbk-ub-unibz Submission To The Epic-kitchens-100 Multi-instance Retrieval Challenge 2022 (2022)0.00
• Egocentric Video-language Pretraining @ EPIC-KITCHENS-100 Multi-instance Retrieval Challenge 2022 (2022)4.83
• A Quadruplet Loss For Enforcing Semantically Coherent Embeddings In Multi-output Classification Problems (2020)6.77
• Unified Loss Of Pair Similarity Optimization For Vision-language Retrieval (2022)0.00
• Intra-modal Constraint Loss For Image-text Retrieval (2022)8.33