All In One Framework For Multimodal Re-identification In The Wild

Abstract

In Re-identification (ReID), recent advancements yield noteworthy progress in both unimodal and cross-modal retrieval tasks. However, the challenge persists in developing a unified framework that could effectively handle varying multimodal data, including RGB, infrared, sketches, and textual information. Additionally, the emergence of large-scale models shows promising performance in various vision tasks but the foundation model in ReID is still blank. In response to these challenges, a novel multimodal learning paradigm for ReID is introduced, referred to as All-in-One (AIO), which harnesses a frozen pre-trained big model as an encoder, enabling effective multimodal retrieval without additional fine-tuning. The diverse multimodal data in AIO are seamlessly tokenized into a unified space, allowing the modality-shared frozen encoder to extract identity-consistent features comprehensively across all modalities. Furthermore, a meticulously crafted ensemble of cross-modality heads is desig

Stats

• citations23
• S2 citations—
• github stars0
• HF likes0
• heat score10.35
• arxiv keyli2024all

Related papers

• Mdreid: Modality-decoupled Learning For Any-to-any Multi-modal Object Re-identification (2025)0.00
• Flexireid: Adaptive Mixture Of Expert For Multi-modal Person Re-identification (2025)0.00
• Mix-modality Person Re-identification: A New And Practical Paradigm (2024)6.34
• Bridging The Gap: Multi-level Cross-modality Joint Alignment For Visible-infrared Person Re-identification (2023)11.29
• Multi-memory Matching For Unsupervised Visible-infrared Person Re-identification (2024)12.02
• Instruct-reid++: Towards Universal Purpose Instruction-guided Person Re-identification (2024)9.13
• Learning Modal-invariant And Temporal-memory For Video-based Visible-infrared Person Re-identification (2022)14.23
• Implicit Discriminative Knowledge Learning For Visible-infrared Person Re-identification (2024)16.19