Conditional Variational Autoencoder With Adversarial Learning For End-to-end Text-to-speech

Abstract

Several recent end-to-end text-to-speech (TTS) models enabling single-stage training and parallel sampling have been proposed, but their sample quality does not match that of two-stage TTS systems. In this work, we present a parallel end-to-end TTS method that generates more natural sounding audio than current two-stage models. Our method adopts variational inference augmented with normalizing flows and an adversarial training process, which improves the expressive power of generative modeling. We also propose a stochastic duration predictor to synthesize speech with diverse rhythms from input text. With the uncertainty modeling over latent variables and the stochastic duration predictor, our method expresses the natural one-to-many relationship in which a text input can be spoken in multiple ways with different pitches and rhythms. A subjective human evaluation (mean opinion score, or MOS) on the LJ Speech, a single speaker dataset, shows that our method outperforms the best publicly

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykim2021conditional

Related papers

• End-to-end Adversarial Text-to-speech (2020)0.00
• Text-to-speech Synthesis Based On Latent Variable Conversion Using Diffusion Probabilistic Model And Variational Autoencoder (2022)0.00
• VITS2: Improving Quality And Efficiency Of Single-stage Text-to-speech With Adversarial Learning And Architecture Design (2023)12.40
• Parallel Tacotron: Non-autoregressive And Controllable TTS (2020)12.54
• Autotts: End-to-end Text-to-speech Synthesis Through Differentiable Duration Modeling (2022)0.00
• Central Kurdish Text-to-speech Synthesis With Novel End-to-end Transformer Training (2024)0.00
• Continuous Autoregressive Modeling With Stochastic Monotonic Alignment For Speech Synthesis (2025)0.00
• Expediting TTS Synthesis With Adversarial Vocoding (2019)6.77