Preference-based Training Framework For Automatic Speech Quality Assessment Using Deep Neural Network

Abstract

One objective of Speech Quality Assessment (SQA) is to estimate the ranks of synthetic speech systems. However, recent SQA models are typically trained using low-precision direct scores such as mean opinion scores (MOS) as the training objective, which is not straightforward to estimate ranking. Although it is effective for predicting quality scores of individual sentences, this approach does not account for speech and system preferences when ranking multiple systems. We propose a training framework of SQA models that can be trained with only preference scores derived from pairs of MOS to improve ranking prediction. Our experiment reveals conditions where our framework works the best in terms of pair generation, aggregation functions to derive system score from utterance preferences, and threshold functions to determine preference from a pair of MOS. Our results demonstrate that our proposed method significantly outperforms the baseline model in Spearman's Rank Correlation Coefficient.

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keyhu2023preference

Related papers

• Attention-based Speech Enhancement Using Human Quality Perception Modelling (2023)0.00
• Non-intrusive Speech Quality Assessment Using Neural Networks (2019)13.74
• Comparison Of Speech Representations For Automatic Quality Estimation In Multi-speaker Text-to-speech Synthesis (2020)0.00
• Using RLHF To Align Speech Enhancement Approaches To Mean-opinion Quality Scores (2024)0.00
• Attentivemos: A Lightweight Attention-only Model For Speech Quality Prediction (2024)3.58
• Learning To Maximize Speech Quality Directly Using MOS Prediction For Neural Text-to-speech (2020)7.81
• A Comparison Of Deep Learning MOS Predictors For Speech Synthesis Quality (2022)6.34
• SCOREQ: Speech Quality Assessment With Contrastive Regression (2024)8.09