Trustser: On The Trustworthiness Of Fine-tuning Pre-trained Speech Embeddings For Speech Emotion Recognition

Abstract

Recent studies have explored the use of pre-trained embeddings for speech emotion recognition (SER), achieving comparable performance to conventional methods that rely on low-level knowledge-inspired acoustic features. These embeddings are often generated from models trained on large-scale speech datasets using self-supervised or weakly-supervised learning objectives. Despite the significant advancements made in SER through the use of pre-trained embeddings, there is a limited understanding of the trustworthiness of these methods, including privacy breaches, unfair performance, vulnerability to adversarial attacks, and computational cost, all of which may hinder the real-world deployment of these systems. In response, we introduce TrustSER, a general framework designed to evaluate the trustworthiness of SER systems using deep learning methods, with a focus on privacy, safety, fairness, and sustainability, offering unique insights into future research in the field of SER. Our code is pu

Stats

• citations7
• S2 citations—
• github stars13
• HF likes0
• heat score9.07
• arxiv keyfeng2023trustser

Related papers

• A Comparative Study Of Pre-trained Speech And Audio Embeddings For Speech Emotion Recognition (2023)0.00
• Transforming The Embeddings: A Lightweight Technique For Speech Emotion Recognition Tasks (2023)7.50
• Towards Interpretable And Transferable Speech Emotion Recognition: Latent Representation Based Analysis Of Features, Methods And Corpora (2021)0.00
• Active Learning Based Fine-tuning Framework For Speech Emotion Recognition (2023)6.34
• SER Evals: In-domain And Out-of-domain Benchmarking For Speech Emotion Recognition (2024)4.52
• Probing Speech Emotion Recognition Transformers For Linguistic Knowledge (2022)9.59
• Enrolment-based Personalisation For Improving Individual-level Fairness In Speech Emotion Recognition (2024)3.58
• Leveraging Speech PTM, Text LLM, And Emotional TTS For Speech Emotion Recognition (2023)10.97