Leveraging Label Potential For Enhanced Multimodal Emotion Recognition

Abstract

Multimodal emotion recognition (MER) seeks to integrate various modalities to predict emotional states accurately. However, most current research focuses solely on the fusion of audio and text features, overlooking the valuable information in emotion labels. This oversight could potentially hinder the performance of existing methods, as emotion labels harbor rich, insightful information that could significantly aid MER. We introduce a novel model called Label Signal-Guided Multimodal Emotion Recognition (LSGMER) to overcome this limitation. This model aims to fully harness the power of emotion label information to boost the classification accuracy and stability of MER. Specifically, LSGMER employs a Label Signal Enhancement module that optimizes the representation of modality features by interacting with audio and text features through label embeddings, enabling it to capture the nuances of emotions precisely. Furthermore, we propose a Joint Objective Optimization(JOO) approach to enha

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshao2025leveraging

Related papers

• Enhancing Modal Fusion By Alignment And Label Matching For Multimodal Emotion Recognition (2024)6.34
• Audio-guided Fusion Techniques For Multimodal Emotion Analysis (2024)4.52
• MMER: Multimodal Multi-task Learning For Speech Emotion Recognition (2022)10.07
• MIAR: Modality Interaction And Alignment Representation Fuison For Multimodal Emotion (2026)0.00
• Bemerc: Behavior-aware Mllm-based Framework For Multimodal Emotion Recognition In Conversation (2025)0.00
• Beyond Silent Letters: Amplifying Llms In Emotion Recognition With Vocal Nuances (2024)9.23
• Effmulti: Efficiently Modeling Complex Multimodal Interactions For Emotion Analysis (2022)0.00
• Jointly Fine-tuning "bert-like" Self Supervised Models To Improve Multimodal Speech Emotion Recognition (2020)13.74