Voice Transformer Network: Sequence-to-sequence Voice Conversion Using Transformer With Text-to-speech Pretraining

Abstract

We introduce a novel sequence-to-sequence (seq2seq) voice conversion (VC) model based on the Transformer architecture with text-to-speech (TTS) pretraining. Seq2seq VC models are attractive owing to their ability to convert prosody. While seq2seq models based on recurrent neural networks (RNNs) and convolutional neural networks (CNNs) have been successfully applied to VC, the use of the Transformer network, which has shown promising results in various speech processing tasks, has not yet been investigated. Nonetheless, their data-hungry property and the mispronunciation of converted speech make seq2seq models far from practical. To this end, we propose a simple yet effective pretraining technique to transfer knowledge from learned TTS models, which benefit from large-scale, easily accessible TTS corpora. VC models initialized with such pretrained model parameters are able to generate effective hidden representations for high-fidelity, highly intelligible converted speech. Experimental

Stats

• citations56
• S2 citations—
• github stars0
• HF likes0
• heat score13.17
• arxiv keyhuang2019voice

Related papers

• Transfer Learning From Speech Synthesis To Voice Conversion With Non-parallel Training Data (2020)12.74
• Bootstrapping Non-parallel Voice Conversion From Speaker-adaptive Text-to-speech (2019)8.35
• Atts2s-vc: Sequence-to-sequence Voice Conversion With Attention And Context Preservation Mechanisms (2018)14.15
• Emotional Voice Conversion Using Multitask Learning With Text-to-speech (2019)0.00
• Convs2s-vc: Fully Convolutional Sequence-to-sequence Voice Conversion (2018)12.68
• Neural Speech Synthesis With Transformer Network (2018)19.95
• Voice Conversion Using Sequence-to-sequence Learning Of Context Posterior Probabilities (2017)11.39
• Hierarchical Sequence To Sequence Voice Conversion With Limited Data (2019)0.00