Generating Diverse And Natural Text-to-speech Samples Using A Quantized Fine-grained VAE And Auto-regressive Prosody Prior
2020 · Guangzhi Sun, Yu Zhang, Ron J. Weiss, et al.
Abstract
Recent neural text-to-speech (TTS) models with fine-grained latent features enable precise control of the prosody of synthesized speech. Such models typically incorporate a fine-grained variational autoencoder (VAE) structure, extracting latent features at each input token (e.g., phonemes). However, generating samples with the standard VAE prior often results in unnatural and discontinuous speech, with dramatic prosodic variation between tokens. This paper proposes a sequential prior in a discrete latent space which can generate more naturally sounding samples. This is accomplished by discretizing the latent features using vector quantization (VQ), and separately training an autoregressive (AR) prior model over the result. We evaluate the approach using listening tests, objective metrics of automatic speech recognition (ASR) performance, and measurements of prosody attributes. Experimental results show that the proposed model significantly improves the naturalness in random sample gene
Authors
(none)
Tags
Stats
Related papers
- Using Generative Modelling To Produce Varied Intonation For Speech Synthesis (2019)7.81
- VAENAR-TTS: Variational Auto-encoder Based Non-autoregressive Text-to-speech Synthesis (2021)7.50
- Prosospeech: Enhancing Prosody With Quantized Vector Pre-training In Text-to-speech (2022)10.61
- DQR-TTS: Semi-supervised Text-to-speech Synthesis With Dynamic Quantized Representation (2023)2.26
- Continuous Autoregressive Modeling With Stochastic Monotonic Alignment For Speech Synthesis (2025)0.00
- Unsupervised Quantized Prosody Representation For Controllable Speech Synthesis (2022)4.52
- Conditional Variational Autoencoder With Adversarial Learning For End-to-end Text-to-speech (2021)0.00
- Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languages (2020)9.59