Statistical Parametric Speech Synthesis Using Generative Adversarial Networks Under A Multi-task Learning Framework

Abstract

In this paper, we aim at improving the performance of synthesized speech in statistical parametric speech synthesis (SPSS) based on a generative adversarial network (GAN). In particular, we propose a novel architecture combining the traditional acoustic loss function and the GAN's discriminative loss under a multi-task learning (MTL) framework. The mean squared error (MSE) is usually used to estimate the parameters of deep neural networks, which only considers the numerical difference between the raw audio and the synthesized one. To mitigate this problem, we introduce the GAN as a second task to determine if the input is a natural speech with specific conditions. In this MTL framework, the MSE optimization improves the stability of GAN, and at the same time GAN produces samples with a distribution closer to natural speech. Listening tests show that the multi-task architecture can generate more natural speech that satisfies human perception than the conventional methods.

Stats

• citations22
• S2 citations—
• github stars0
• HF likes0
• heat score10.21
• arxiv keyyang2017statistical

Related papers

• Statistical Parametric Speech Synthesis Incorporating Generative Adversarial Networks (2017)16.21
• Multi-task Adversarial Training Algorithm For Multi-speaker Neural Text-to-speech (2022)0.00
• Ganspeech: Adversarial Training For High-fidelity Multi-speaker Speech Synthesis (2021)10.07
• Multi-spectrogan: High-diversity And High-fidelity Spectrogram Generation With Adversarial Style Combination For Speech Synthesis (2020)0.00
• Multi-task Wavenet: A Multi-task Generative Model For Statistical Parametric Speech Synthesis Without Fundamental Frequency Conditions (2018)8.09
• High Fidelity Speech Synthesis With Adversarial Networks (2019)0.00
• Generative Adversarial Network-based Glottal Waveform Model For Statistical Parametric Speech Synthesis (2019)10.35
• Multi-metric Optimization Using Generative Adversarial Networks For Near-end Speech Intelligibility Enhancement (2021)8.60