Cross-modal And Multimodal Data Analysis Based On Functional Mapping Of Spectral Descriptors And Manifold Regularization

Abstract

Multimodal manifold modeling methods extend the spectral geometry-aware data analysis to learning from several related and complementary modalities. Most of these methods work based on two major assumptions: 1) there are the same number of homogeneous data samples in each modality, and 2) at least partial correspondences between modalities are given in advance as prior knowledge. This work proposes two new multimodal modeling methods. The first method establishes a general analyzing framework to deal with the multimodal information problem for heterogeneous data without any specific prior knowledge. For this purpose, first, we identify the localities of each manifold by extracting local descriptors via spectral graph wavelet signatures (SGWS). Then, we propose a manifold regularization framework based on the functional mapping between SGWS descriptors (FMBSD) for finding the pointwise correspondences. The second method is a manifold regularized multimodal classification based on pointw

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybehmanesh2021cross

Related papers

• Cross-modal Manifold Learning For Cross-modal Retrieval (2016)0.00
• Latent Functional Maps: A Spectral Framework For Representation Alignment (2024)3.58
• On The Spectral Geometry Of Cross-modal Representations: A Functional Map Diagnostic For Multimodal Alignment (2026)0.00
• An Application Of Manifold Learning In Global Shape Descriptors (2019)5.84
• Metric Learning On Manifolds (2019)0.00
• Modal-aware Features For Multimodal Hashing (2019)0.00
• A Discriminative Vectorial Framework For Multi-modal Feature Representation (2021)8.60
• Quadric Hypersurface Intersection For Manifold Learning In Feature Space (2021)0.00