Multitask Vocal Burst Modeling With Resnets And Pre-trained Paralinguistic Conformers

Abstract

This technical report presents the modeling approaches used in our submission to the ICML Expressive Vocalizations Workshop & Competition multitask track (ExVo-MultiTask). We first applied image classification models of various sizes on mel-spectrogram representations of the vocal bursts, as is standard in sound event detection literature. Results from these models show an increase of 21.24% over the baseline system with respect to the harmonic mean of the task metrics, and comprise our team's main submission to the MultiTask track. We then sought to characterize the headroom in the MultiTask track by applying a large pre-trained Conformer model that previously achieved state-of-the-art results on paralinguistic tasks like speech emotion recognition and mask detection. We additionally investigated the relationship between the sub-tasks of emotional expression, country of origin, and age prediction, and discovered that the best performing models are trained as single-task models, questi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybelanich2022multitask

Related papers

• Self-supervision And Learnable Strfs For Age, Emotion, And Country Prediction (2022)0.00
• An Efficient Multitask Learning Architecture For Affective Vocal Burst Analysis (2022)0.00
• Dynamic Restrained Uncertainty Weighting Loss For Multitask Learning Of Vocal Expression (2022)0.00
• Burst2vec: An Adversarial Multi-task Approach For Predicting Emotion, Age, And Origin From Vocal Bursts (2022)0.00
• Self-supervised Attention Networks And Uncertainty Loss Weighting For Multi-task Emotion Recognition On Vocal Bursts (2022)0.00
• Self-relation Attention And Temporal Awareness For Emotion Recognition Via Vocal Burst (2022)4.18
• Classification Of Vocal Bursts For ACII 2022 A-vb-type Competition Using Convolutional Neural Networks And Deep Acoustic Embeddings (2022)0.00
• Advancing Audio Emotion And Intent Recognition With Large Pre-trained Models And Bayesian Inference (2023)5.24