Speech Emotion Recognition With Multiscale Area Attention And Data Augmentation

Abstract

In Speech Emotion Recognition (SER), emotional characteristics often appear in diverse forms of energy patterns in spectrograms. Typical attention neural network classifiers of SER are usually optimized on a fixed attention granularity. In this paper, we apply multiscale area attention in a deep convolutional neural network to attend emotional characteristics with varied granularities and therefore the classifier can benefit from an ensemble of attentions with different scales. To deal with data sparsity, we conduct data augmentation with vocal tract length perturbation (VTLP) to improve the generalization capability of the classifier. Experiments are carried out on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) dataset. We achieved 79.34% weighted accuracy (WA) and 77.54% unweighted accuracy (UA), which, to the best of our knowledge, is the state of the art on this dataset.

Stats

• citations65
• S2 citations—
• github stars0
• HF likes0
• heat score13.65
• arxiv keyxu2021speech

Related papers

• Attention Based Fully Convolutional Network For Speech Emotion Recognition (2018)15.25
• Hybrid Data Augmentation And Deep Attention-based Dilated Convolutional-recurrent Neural Networks For Speech Emotion Recognition (2021)12.81
• Speech Emotion Recognition Via Cnn-transformer And Multidimensional Attention Mechanism (2024)0.00
• Improved Speech Emotion Recognition Using Transfer Learning And Spectrogram Augmentation (2021)12.74
• Speech Emotion Recognition With Co-attention Based Multi-level Acoustic Information (2022)16.17
• Multi-window Data Augmentation Approach For Speech Emotion Recognition (2020)0.00
• Supervised Contrastive Learning With Nearest Neighbor Search For Speech Emotion Recognition (2023)7.16
• Improving Speech Emotion Recognition Through Focus And Calibration Attention Mechanisms (2022)0.00