Conversational Emotion Analysis Via Attention Mechanisms

Abstract

Different from the emotion recognition in individual utterances, we propose a multimodal learning framework using relation and dependencies among the utterances for conversational emotion analysis. The attention mechanism is applied to the fusion of the acoustic and lexical features. Then these fusion representations are fed into the self-attention based bi-directional gated recurrent unit (GRU) layer to capture long-term contextual information. To imitate real interaction patterns of different speakers, speaker embeddings are also utilized as additional inputs to distinguish the speaker identities during conversational dialogs. To verify the effectiveness of the proposed method, we conduct experiments on the IEMOCAP database. Experimental results demonstrate that our method shows absolute 2.42% performance improvement over the state-of-the-art strategies.

Stats

• citations23
• S2 citations—
• github stars0
• HF likes0
• heat score10.35
• arxiv keylian2019conversational

Related papers

• Speech Emotion Recognition Using Multi-hop Attention Mechanism (2019)14.58
• Attentive Modality Hopping Mechanism For Speech Emotion Recognition (2019)0.00
• Gatedxlstm: A Multimodal Affective Computing Approach For Emotion Recognition In Conversations (2025)0.00
• Exploring Attention Mechanisms For Multimodal Emotion Recognition In An Emergency Call Center Corpus (2023)8.09
• Speech Emotion Recognition With Multiscale Area Attention And Data Augmentation (2021)13.65
• Attention Based Fully Convolutional Network For Speech Emotion Recognition (2018)15.25
• Group Gated Fusion On Attention-based Bidirectional Alignment For Multimodal Emotion Recognition (2022)11.39
• Effmulti: Efficiently Modeling Complex Multimodal Interactions For Emotion Analysis (2022)0.00