Self-supervision And Learnable Strfs For Age, Emotion, And Country Prediction

Abstract

This work presents a multitask approach to the simultaneous estimation of age, country of origin, and emotion given vocal burst audio for the 2022 ICML Expressive Vocalizations Challenge ExVo-MultiTask track. The method of choice utilized a combination of spectro-temporal modulation and self-supervised features, followed by an encoder-decoder network organized in a multitask paradigm. We evaluate the complementarity between the tasks posed by examining independent task-specific and joint models, and explore the relative strengths of different feature sets. We also introduce a simple score fusion mechanism to leverage the complementarity of different feature sets for this task. We find that robust data preprocessing in conjunction with score fusion over spectro-temporal receptive field and HuBERT models achieved our best ExVo-MultiTask test score of 0.412.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysharma2022self

Related papers

• Jointly Predicting Emotion, Age, And Country Using Pre-trained Acoustic Embedding (2022)6.77
• Burst2vec: An Adversarial Multi-task Approach For Predicting Emotion, Age, And Origin From Vocal Bursts (2022)0.00
• Comparing Supervised And Self-supervised Embedding For Exvo Multi-task Learning Track (2022)0.00
• Multitask Vocal Burst Modeling With Resnets And Pre-trained Paralinguistic Conformers (2022)0.00
• Dynamic Restrained Uncertainty Weighting Loss For Multitask Learning Of Vocal Expression (2022)0.00
• Self-relation Attention And Temporal Awareness For Emotion Recognition Via Vocal Burst (2022)4.18
• Self-supervised Attention Networks And Uncertainty Loss Weighting For Multi-task Emotion Recognition On Vocal Bursts (2022)0.00
• Speech Emotion: Investigating Model Representations, Multi-task Learning And Knowledge Distillation (2022)6.34