Hierarchical Sequence To Sequence Voice Conversion With Limited Data

Abstract

We present a voice conversion solution using recurrent sequence to sequence modeling for DNNs. Our solution takes advantage of recent advances in attention based modeling in the fields of Neural Machine Translation (NMT), Text-to-Speech (TTS) and Automatic Speech Recognition (ASR). The problem consists of converting between voices in a parallel setting when \{\it \(<\)source,target\(>\)\} audio pairs are available. Our seq2seq architecture makes use of a hierarchical encoder to summarize input audio frames. On the decoder side, we use an attention based architecture used in recent TTS works. Since there is a dearth of large multispeaker voice conversion databases needed for training DNNs, we resort to training the network with a large single speaker dataset as an autoencoder. This is then adapted for the smaller multispeaker voice conversion datasets available for voice conversion. In contrast with other voice conversion works that use \(F_0\), duration and linguistic features, our sys

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynarayanan2019hierarchical

Related papers

• Non-parallel Sequence-to-sequence Voice Conversion With Disentangled Linguistic And Speaker Representations (2019)14.02
• Singing Voice Conversion With Non-parallel Data (2019)9.59
• Voice Transformer Network: Sequence-to-sequence Voice Conversion Using Transformer With Text-to-speech Pretraining (2019)13.17
• Any-to-one Sequence-to-sequence Voice Conversion Using Self-supervised Discrete Speech Representations (2020)0.00
• Transfer Learning From Speech Synthesis To Voice Conversion With Non-parallel Training Data (2020)12.74
• Sequence-to-sequence Acoustic Modeling For Voice Conversion (2018)14.97
• Convs2s-vc: Fully Convolutional Sequence-to-sequence Voice Conversion (2018)12.68
• Any-to-many Voice Conversion With Location-relative Sequence-to-sequence Modeling (2020)14.02