Maskedspeech: Context-aware Speech Synthesis With Masking Strategy

Abstract

Humans often speak in a continuous manner which leads to coherent and consistent prosody properties across neighboring utterances. However, most state-of-the-art speech synthesis systems only consider the information within each sentence and ignore the contextual semantic and acoustic features. This makes it inadequate to generate high-quality paragraph-level speech which requires high expressiveness and naturalness. To synthesize natural and expressive speech for a paragraph, a context-aware speech synthesis system named MaskedSpeech is proposed in this paper, which considers both contextual semantic and acoustic features. Inspired by the masking strategy in the speech editing research, the acoustic features of the current sentence are masked out and concatenated with those of contextual speech, and further used as additional model input. The phoneme encoder takes the concatenated phoneme sequence from neighboring sentences as input and learns fine-grained semantic information from co

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyzhang2022maskedspeech

Related papers

• Contextspeech: Expressive And Efficient Text-to-speech For Paragraph Reading (2023)5.84
• Campnet: Context-aware Mask Prediction For End-to-end Text-based Speech Editing (2022)8.60
• Joint Semantic Knowledge Distillation And Masked Acoustic Modeling For Full-band Speech Restoration With Improved Intelligibility (2024)4.52
• Maskgct: Zero-shot Text-to-speech With Masked Generative Codec Transformer (2024)7.98
• Effective Decoder Masking For Transformer Based End-to-end Speech Recognition (2020)0.00
• Syncspeech: Efficient And Low-latency Text-to-speech Based On Temporal Masked Transformer (2025)0.00
• Exploiting Deep Sentential Context For Expressive End-to-end Speech Synthesis (2020)5.84
• Multimodal Speech Recognition With Unstructured Audio Masking (2020)0.00