Generative Data Augmentation Guided By Triplet Loss For Speech Emotion Recognition

Abstract

Speech Emotion Recognition (SER) is crucial for human-computer interaction but still remains a challenging problem because of two major obstacles: data scarcity and imbalance. Many datasets for SER are substantially imbalanced, where data utterances of one class (most often Neutral) are much more frequent than those of other classes. Furthermore, only a few data resources are available for many existing spoken languages. To address these problems, we exploit a GAN-based augmentation model guided by a triplet network, to improve SER performance given imbalanced and insufficient training data. We conduct experiments and demonstrate: 1) With a highly imbalanced dataset, our augmentation strategy significantly improves the SER performance (+8% recall score compared with the baseline). 2) Moreover, in a cross-lingual benchmark, where we train a model with enough source language utterances but very few target language utterances (around 50 in our experiments), our augmentation strategy bring

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keywang2022generative

Related papers

• Augmenting Generative Adversarial Networks For Speech Emotion Recognition (2020)10.85
• Improved Speech Emotion Recognition Using Transfer Learning And Spectrogram Augmentation (2021)12.74
• Hybrid Data Augmentation And Deep Attention-based Dilated Convolutional-recurrent Neural Networks For Speech Emotion Recognition (2021)12.81
• Unsupervised Adversarial Domain Adaptation For Cross-lingual Speech Emotion Recognition (2019)12.74
• Foundation Model Assisted Automatic Speech Emotion Recognition: Transcribing, Annotating, And Augmenting (2023)0.00
• Generative Emotional AI For Speech Emotion Recognition: The Case For Synthetic Emotional Speech Augmentation (2023)11.19
• A Preliminary Study On Augmenting Speech Emotion Recognition Using A Diffusion Model (2023)0.00
• Copypaste: An Augmentation Method For Speech Emotion Recognition (2020)11.39