Cross-modal Center Loss

Abstract

Cross-modal retrieval aims to learn discriminative and modal-invariant features for data from different modalities. Unlike the existing methods which usually learn from the features extracted by offline networks, in this paper, we propose an approach to jointly train the components of cross-modal retrieval framework with metadata, and enable the network to find optimal features. The proposed end-to-end framework is updated with three loss functions: 1) a novel cross-modal center loss to eliminate cross-modal discrepancy, 2) cross-entropy loss to maximize inter-class variations, and 3) mean-square-error loss to reduce modality variations. In particular, our proposed cross-modal center loss minimizes the distances of features from objects belonging to the same class across all modalities. Extensive experiments have been conducted on the retrieval tasks across multi-modalities, including 2D image, 3D point cloud, and mesh data. The proposed framework significantly outperforms the state-of

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyjing2020cross

Related papers

• Preserving Semantic Neighborhoods For Robust Cross-modal Retrieval (2020)10.07
• Discriminative Semantic Transitive Consistency For Cross-modal Learning (2021)0.00
• End-to-end Cross-modality Retrieval With CCA Projections And Pairwise Ranking Loss (2017)14.68
• Intra-modal Constraint Loss For Image-text Retrieval (2022)8.33
• Adversarial Cross-modal Retrieval Via Learning And Transferring Single-modal Similarities (2019)8.60
• Cross-modal Retrieval: A Systematic Review Of Methods And Future Directions (2023)12.81
• Neighborretr: Balancing Hub Centrality In Cross-modal Retrieval (2025)4.17
• Triplet-center Loss For Multi-view 3D Object Retrieval (2018)18.70