SPEAKER VGG CCT: Cross-corpus Speech Emotion Recognition With Speaker Embedding And Vision Transformers

Abstract

In recent years, Speech Emotion Recognition (SER) has been investigated mainly transforming the speech signal into spectrograms that are then classified using Convolutional Neural Networks pretrained on generic images and fine tuned with spectrograms. In this paper, we start from the general idea above and develop a new learning solution for SER, which is based on Compact Convolutional Transformers (CCTs) combined with a speaker embedding. With CCTs, the learning power of Vision Transformers (ViT) is combined with a diminished need for large volume of data as made possible by the convolution. This is important in SER, where large corpora of data are usually not available. The speaker embedding allows the network to extract an identity representation of the speaker, which is then integrated by means of a self-attention mechanism with the features that the CCT extracts from the spectrogram. Overall, the solution is capable of operating in real-time showing promising results in a cross-co

Stats

• citations0
• S2 citations—
• github stars25
• HF likes0
• heat score2.83
• arxiv keyarezzo2022speaker

Related papers

• Transforming The Embeddings: A Lightweight Technique For Speech Emotion Recognition Tasks (2023)7.50
• Ctl-mtnet: A Novel Capsnet And Transfer Learning-based Mixed Task Net For The Single-corpus And Cross-corpus Speech Emotion Recognition (2022)10.21
• End-to-end Transfer Learning For Speaker-independent Cross-language And Cross-corpus Speech Emotion Recognition (2023)0.00
• Improved Speech Emotion Recognition Using Transfer Learning And Spectrogram Augmentation (2021)12.74
• Leveraging Cross-attention Transformer And Multi-feature Fusion For Cross-linguistic Speech Emotion Recognition (2025)4.52
• Dawn Of The Transformer Era In Speech Emotion Recognition: Closing The Valence Gap (2022)18.59
• Speaker Emotion Recognition: Leveraging Self-supervised Models For Feature Extraction Using Wav2vec2 And Hubert (2024)0.00
• Speech Emotion Recognition Via Cnn-transformer And Multidimensional Attention Mechanism (2024)0.00