Hierarchical Prosody Modeling For Non-autoregressive Speech Synthesis

Abstract

Prosody modeling is an essential component in modern text-to-speech (TTS) frameworks. By explicitly providing prosody features to the TTS model, the style of synthesized utterances can thus be controlled. However, predicting natural and reasonable prosody at inference time is challenging. In this work, we analyzed the behavior of non-autoregressive TTS models under different prosody-modeling settings and proposed a hierarchical architecture, in which the prediction of phoneme-level prosody features are conditioned on the word-level prosody features. The proposed method outperforms other competitors in terms of audio quality and prosody naturalness in our objective and subjective evaluation.

Stats

• citations21
• S2 citations—
• github stars0
• HF likes0
• heat score10.07
• arxiv keychien2020hierarchical

Related papers

• Hierarchical Prosody Modeling And Control In Non-autoregressive Parallel Neural TTS (2021)8.35
• Fully-hierarchical Fine-grained Prosody Modeling For Interpretable Speech Synthesis (2020)13.28
• Towards Expressive Zero-shot Speech Synthesis With Hierarchical Prosody Modeling (2024)4.52
• Hignn-tts: Hierarchical Prosody Modeling With Graph Neural Networks For Expressive Long-form TTS (2023)5.84
• Diffstyletts: Diffusion-based Hierarchical Prosody Modeling For Text-to-speech With Diverse And Controllable Styles (2024)0.00
• Applying Syntax\(\unicode{x2013}\)prosody Mapping Hypothesis And Prosodic Well-formedness Constraints To Neural Sequence-to-sequence Speech Synthesis (2022)0.00
• Improving Prosody For Cross-speaker Style Transfer By Semi-supervised Style Extractor And Hierarchical Modeling In Speech Synthesis (2023)7.50
• Controllable Neural Text-to-speech Synthesis Using Intuitive Prosodic Features (2020)11.76