Semi-supervised Training For Improving Data Efficiency In End-to-end Speech Synthesis

Abstract

Although end-to-end text-to-speech (TTS) models such as Tacotron have shown excellent results, they typically require a sizable set of high-quality <text, audio> pairs for training, which are expensive to collect. In this paper, we propose a semi-supervised training framework to improve the data efficiency of Tacotron. The idea is to allow Tacotron to utilize textual and acoustic knowledge contained in large, publicly-available text and speech corpora. Importantly, these external data are unpaired and potentially noisy. Specifically, first we embed each word in the input text into word vectors and condition the Tacotron encoder on them. We then use an unpaired speech corpus to pre-train the Tacotron decoder in the acoustic domain. Finally, we fine-tune the model using available paired data. We demonstrate that the proposed framework enables Tacotron to generate intelligible speech using less than half an hour of paired training data.

Stats

• citations58
• S2 citations—
• github stars0
• HF likes0
• heat score13.28
• arxiv keychung2018semi

Related papers

• Tacotron: Towards End-to-end Speech Synthesis (2017)0.00
• Semi-supervised Learning For Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation (2020)5.24
• Towards Transfer Learning For End-to-end Speech Synthesis From Deep Pre-trained Language Models (2019)0.00
• Non-attentive Tacotron: Robust And Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling (2020)0.00
• Parallel Tacotron: Non-autoregressive And Controllable TTS (2020)12.54
• Non-autoregressive TTS With Explicit Duration Modelling For Low-resource Highly Expressive Speech (2021)8.82
• Efficienttts: An Efficient And High-quality Text-to-speech Architecture (2020)0.00
• Towards End-to-end Prosody Transfer For Expressive Speech Synthesis With Tacotron (2018)0.00