Generalized Multi-view Embedding For Visual Recognition And Cross-modal Retrieval

Abstract

In this paper, the problem of multi-view embedding from different visual cues and modalities is considered. We propose a unified solution for subspace learning methods using the Rayleigh quotient, which is extensible for multiple views, supervised learning, and non-linear embeddings. Numerous methods including Canonical Correlation Analysis, Partial Least Sqaure regression and Linear Discriminant Analysis are studied using specific intrinsic and penalty graphs within the same framework. Non-linear extensions based on kernels and (deep) neural networks are derived, achieving better performance than the linear ones. Moreover, a novel Multi-view Modular Discriminant Analysis (MvMDA) is proposed by taking the view difference into consideration. We demonstrate the effectiveness of the proposed multi-view embedding methods on visual object recognition and cross-modal image retrieval, and obtain superior results in both applications compared to related methods.

Stats

• citations90
• S2 citations—
• github stars0
• HF likes0
• heat score14.69
• arxiv keycao2016generalized

Related papers

• Learning Discriminative Hashing Codes For Cross-modal Retrieval Based On Multi-view Features (2018)3.58
• Unified Representation Learning For Cross Model Compatibility (2020)5.24
• Dynamic Visual Semantic Sub-embeddings And Fast Re-ranking (2023)0.00
• Universal Vision-language Dense Retrieval: Learning A Unified Representation Space For Multi-modal Retrieval (2022)3.45
• Cross-view Image Retrieval -- Ground To Aerial Image Retrieval Through Deep Learning (2020)5.24
• Self-weighted Multiview Metric Learning By Maximizing The Cross Correlations (2019)0.00
• Vlm2geovec: Toward Universal Multimodal Embeddings For Remote Sensing (2025)0.00
• Discriminative Cross-view Binary Representation Learning (2018)4.52