Learning To Maximize Speech Quality Directly Using MOS Prediction For Neural Text-to-speech

Abstract

Although recent neural text-to-speech (TTS) systems have achieved high-quality speech synthesis, there are cases where a TTS system generates low-quality speech, mainly caused by limited training data or information loss during knowledge distillation. Therefore, we propose a novel method to improve speech quality by training a TTS model under the supervision of perceptual loss, which measures the distance between the maximum possible speech quality score and the predicted one. We first pre-train a mean opinion score (MOS) prediction model and then train a TTS model to maximize the MOS of synthesized speech using the pre-trained MOS prediction model. The proposed method can be applied independently regardless of the TTS model architecture or the cause of speech quality degradation and efficiently without increasing the inference time or model complexity. The evaluation results for the MOS and phone error rate demonstrate that our proposed approach improves previous models in terms of bo

Stats

• citations10
• S2 citations—
• github stars0
• HF likes0
• heat score7.81
• arxiv keychoi2020learning

Related papers

• Investigating Content-aware Neural Text-to-speech MOS Prediction Using Prosodic And Linguistic Features (2022)6.34
• Neural MOS Prediction For Synthesized Speech Using Multi-task Learning With Spoofing Detection And Spoofing Type Classification (2020)9.59
• Attention-based Speech Enhancement Using Human Quality Perception Modelling (2023)0.00
• A Comparison Of Deep Learning MOS Predictors For Speech Synthesis Quality (2022)6.34
• Comparison Of Speech Representations For Automatic Quality Estimation In Multi-speaker Text-to-speech Synthesis (2020)0.00
• Ldnet: Unified Listener Dependent Modeling In MOS Prediction For Synthetic Speech (2021)12.74
• Partial Rank Similarity Minimization Method For Quality MOS Prediction Of Unseen Speech Synthesis Systems In Zero-shot And Semi-supervised Setting (2023)2.26
• Towards High-quality Neural TTS For Low-resource Languages By Learning Compact Speech Representations (2022)0.00