Enhancing Word-level Semantic Representation Via Dependency Structure For Expressive Text-to-speech Synthesis

Abstract

Exploiting rich linguistic information in raw text is crucial for expressive text-to-speech (TTS). As large scale pre-trained text representation develops, bidirectional encoder representations from Transformers (BERT) has been proven to embody semantic information and employed to TTS recently. However, original or simply fine-tuned BERT embeddings still cannot provide sufficient semantic knowledge that expressive TTS models should take into account. In this paper, we propose a word-level semantic representation enhancing method based on dependency structure and pre-trained BERT embedding. The BERT embedding of each word is reprocessed considering its specific dependencies and related words in the sentence, to generate more effective semantic representation for TTS. To better utilize the dependency structure, relational gated graph network (RGGN) is introduced to make semantic information flow and aggregate through the dependency structure. The experimental results show that the propos

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhou2021enhancing

Related papers

• RWEN-TTS: Relation-aware Word Encoding Network For Natural Text-to-speech Synthesis (2022)0.00
• Improving Prosody Modelling With Cross-utterance BERT Embeddings For End-to-end Speech Synthesis (2020)10.61
• Towards Transfer Learning For End-to-end Speech Synthesis From Deep Pre-trained Language Models (2019)0.00
• Exploiting Syntactic Features In A Parsed Tree To Improve End-to-end TTS (2019)9.03
• Feature Reinforcement With Word Embedding And Parsing Information In Neural TTS (2019)0.00
• Exploiting Deep Sentential Context For Expressive End-to-end Speech Synthesis (2020)5.84
• Utilizing Neural Transducers For Two-stage Text-to-speech Via Semantic Token Prediction (2024)0.00
• Simple And Effective Multi-sentence TTS With Expressive And Coherent Prosody (2022)7.16