Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions

Abstract

This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps character embeddings to mel-scale spectrograms, followed by a modified WaveNet model acting as a vocoder to synthesize timedomain waveforms from those spectrograms. Our model achieves a mean opinion score (MOS) of \(4.53\) comparable to a MOS of \(4.58\) for professionally recorded speech. To validate our design choices, we present ablation studies of key components of our system and evaluate the impact of using mel spectrograms as the input to WaveNet instead of linguistic, duration, and \(F_0\) features. We further demonstrate that using a compact acoustic intermediate representation enables significant simplification of the WaveNet architecture.

Stats

• citations1620
• S2 citations—
• github stars0
• HF likes0
• heat score24.07
• arxiv keyshen2017natural

Related papers

• Wave-tacotron: Spectrogram-free End-to-end Text-to-speech Synthesis (2020)12.81
• Neural Speech Synthesis With Transformer Network (2018)19.95
• Fastspeech: Fast, Robust And Controllable Text To Speech (2019)0.00
• Parallel Wavenet Conditioned On VAE Latent Vectors (2020)0.00
• Tacotron: Towards End-to-end Speech Synthesis (2017)0.00
• Controllable Neural Text-to-speech Synthesis Using Intuitive Prosodic Features (2020)11.76
• Non-attentive Tacotron: Robust And Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling (2020)0.00
• Utilizing Neural Transducers For Two-stage Text-to-speech Via Semantic Token Prediction (2024)0.00