A Modular End-to-end Multimodal Learning Method For Structured And Unstructured Data

Abstract

Multimodal learning is a rapidly growing research field that has revolutionized multitasking and generative modeling in AI. While much of the research has focused on dealing with unstructured data (e.g., language, images, audio, or video), structured data (e.g., tabular data, time series, or signals) has received less attention. However, many industry-relevant use cases involve or can be benefited from both types of data. In this work, we propose a modular, end-to-end multimodal learning method called MAGNUM, which can natively handle both structured and unstructured data. MAGNUM is flexible enough to employ any specialized unimodal module to extract, compress, and fuse information from all available modalities.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyalessandro2024a

Related papers

• Meta-transformer: A Unified Framework For Multimodal Learning (2023)6.44
• Towards Robust Multimodal Learning In The Open World (2025)0.00
• Multimodal Large Language Models: A Survey (2023)0.00
• Llms Meet Multimodal Generation And Editing: A Survey (2024)5.48
• Efficient Audiovisual Speech Processing Via MUTUD: Multimodal Training And Unimodal Deployment (2025)0.00
• Next-gpt: Any-to-any Multimodal LLM (2023)0.00
• Attribution Regularization For Multimodal Paradigms (2024)0.00
• M2-omni: Advancing Omni-mllm For Comprehensive Modality Support With Competitive Performance (2025)0.00