CPCL: Cross-modal Prototypical Contrastive Learning For Weakly Supervised Text-based Person Retrieval

Abstract

Weakly supervised text-based person retrieval seeks to retrieve images of a target person using textual descriptions, without relying on identity annotations and is more challenging and practical. The primary challenge is the intra-class differences, encompassing intra-modal feature variations and cross-modal semantic gaps. Prior works have focused on instance-level samples and ignored prototypical features of each person which are intrinsic and invariant. Toward this, we propose a Cross-Modal Prototypical Contrastive Learning (CPCL) method. In practice, the CPCL introduces the CLIP model to weakly supervised text-based person retrieval to map visual and textual instances into a shared latent space. Subsequently, the proposed Prototypical Multi-modal Memory (PMM) module captures associations between heterogeneous modalities of image-text pairs belonging to the same person through the Hybrid Cross-modal Matching (HCM) module in a many-to-many mapping fashion. Moreover, the Outlier Pseud

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhao2024cpcl

Related papers

• Up-person: Unified Parameter-efficient Transfer Learning For Text-based Person Retrieval (2025)4.26
• Generalized Contrastive Learning For Universal Multimodal Retrieval (2025)0.00
• Look Before You Leap: Improving Text-based Person Retrieval By Learning A Consistent Cross-modal Common Manifold (2022)15.34
• PC\(^2\): Pseudo-classification Based Pseudo-captioning For Noisy Correspondence Learning In Cross-modal Retrieval (2024)9.23
• Prototype-guided Cross-modal Completion And Alignment For Incomplete Text-based Person Re-identification (2023)6.77
• Improving Text-based Person Search Via Part-level Cross-modal Correspondence (2024)0.00
• Multi-path Exploration And Feedback Adjustment For Text-to-image Person Retrieval (2024)0.00
• TIPCB: A Simple But Effective Part-based Convolutional Baseline For Text-based Person Search (2021)20.24