Enrolment-based Personalisation For Improving Individual-level Fairness In Speech Emotion Recognition

Abstract

The expression of emotion is highly individualistic. However, contemporary speech emotion recognition (SER) systems typically rely on population-level models that adopt a `one-size-fits-all' approach for predicting emotion. Moreover, standard evaluation practices measure performance also on the population level, thus failing to characterise how models work across different speakers. In the present contribution, we present a new method for capitalising on individual differences to adapt an SER model to each new speaker using a minimal set of enrolment utterances. In addition, we present novel evaluation schemes for measuring fairness across different speakers. Our findings show that aggregated evaluation metrics may obfuscate fairness issues on the individual-level, which are uncovered by our evaluation, and that our proposed method can improve performance both in aggregated and disaggregated terms.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keytriantafyllopoulos2024enrolment

Related papers

• Is It Still Fair? Investigating Gender Fairness In Cross-corpus Speech Emotion Recognition (2025)5.24
• Personalized Adaptation With Pre-trained Speech Encoders For Continuous Emotion Recognition (2023)6.34
• Trustser: On The Trustworthiness Of Fine-tuning Pre-trained Speech Embeddings For Speech Emotion Recognition (2023)9.07
• Unsupervised Personalization Of An Emotion Recognition System: The Unique Properties Of The Externalization Of Valence In Speech (2022)9.03
• Active Learning Based Fine-tuning Framework For Speech Emotion Recognition (2023)6.34
• Towards Interpretable And Transferable Speech Emotion Recognition: Latent Representation Based Analysis Of Features, Methods And Corpora (2021)0.00
• Continuous Metric Learning For Transferable Speech Emotion Recognition And Embedding Across Low-resource Languages (2022)0.00
• Exploring Speaker Enrolment For Few-shot Personalisation In Emotional Vocalisation Prediction (2022)0.00