Improving Prosody Modelling With Cross-utterance BERT Embeddings For End-to-end Speech Synthesis
2020 · Guanghui Xu, Wei Song, Zhengchen Zhang, et al.
Abstract
Despite prosody is related to the linguistic information up to the discourse structure, most text-to-speech (TTS) systems only take into account that within each sentence, which makes it challenging when converting a paragraph of texts into natural and expressive speech. In this paper, we propose to use the text embeddings of the neighboring sentences to improve the prosody generation for each utterance of a paragraph in an end-to-end fashion without using any explicit prosody features. More specifically, cross-utterance (CU) context vectors, which are produced by an additional CU encoder based on the sentence embeddings extracted by a pre-trained BERT model, are used to augment the input of the Tacotron2 decoder. Two types of BERT embeddings are investigated, which leads to the use of different CU encoder structures. Experimental results on a Mandarin audiobook dataset and the LJ-Speech English audiobook dataset demonstrate the use of CU information can improve the naturalness and exp
Authors
(none)
Tags
Stats
Related papers
- Paratts: Learning Linguistic And Prosodic Cross-sentence Information In Paragraph-based TTS (2022)8.82
- Text Enhancement For Paragraph Processing In End-to-end Code-switching TTS (2022)0.00
- Simple And Effective Multi-sentence TTS With Expressive And Coherent Prosody (2022)7.16
- Dynamic Prosody Generation For Speech Synthesis Using Linguistics-driven Acoustic Embedding Selection (2019)7.81
- Improving Speech Prosody Of Audiobook Text-to-speech Synthesis With Acoustic And Textual Contexts (2022)7.81
- Phoneme-level BERT For Enhanced Prosody Of Text-to-speech With Grapheme Predictions (2023)0.00
- Improving Multi-speaker TTS Prosody Variance With A Residual Encoder And Normalizing Flows (2021)0.00
- Adversarial Learning Of Intermediate Acoustic Feature For End-to-end Lightweight Text-to-speech (2022)0.00