Predicting Expressive Speaking Style From Text In End-to-end Speech Synthesis

Abstract

Global Style Tokens (GSTs) are a recently-proposed method to learn latent disentangled representations of high-dimensional data. GSTs can be used within Tacotron, a state-of-the-art end-to-end text-to-speech synthesis system, to uncover expressive factors of variation in speaking style. In this work, we introduce the Text-Predicted Global Style Token (TP-GST) architecture, which treats GST combination weights or style embeddings as "virtual" speaking style labels within Tacotron. TP-GST learns to predict stylistic renderings from text alone, requiring neither explicit labels during training nor auxiliary inputs for inference. We show that, when trained on a dataset of expressive speech, our system generates audio with more pitch and energy variation than two state-of-the-art baseline models. We further demonstrate that TP-GSTs can synthesize speech with background noise removed, and corroborate these analyses with positive results on human-rated listener preference audiobook tasks. Fin

Stats

• citations75
• S2 citations—
• github stars0
• HF likes0
• heat score14.11
• arxiv keystanton2018predicting

Related papers

• Style Tokens: Unsupervised Style Modeling, Control And Transfer In End-to-end Speech Synthesis (2018)0.00
• End-to-end Emotional Speech Synthesis Using Style Tokens And Semi-supervised Training (2019)12.87
• Uncovering Latent Style Factors For Expressive Speech Synthesis (2017)0.00
• Styletts: A Style-based Generative Model For Natural And Diverse Text-to-speech Synthesis (2022)10.97
• End-to-end Text-to-speech Based On Latent Representation Of Speaking Styles Using Spontaneous Dialogue (2022)8.35
• Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarios (2021)6.77
• Expressive Text-to-speech Using Style Tag (2021)10.85
• Expressive TTS Training With Frame And Style Reconstruction Loss (2020)12.74