Sequence-to-sequence Acoustic Modeling For Voice Conversion

Abstract

In this paper, a neural network named Sequence-to-sequence ConvErsion NeTwork (SCENT) is presented for acoustic modeling in voice conversion. At training stage, a SCENT model is estimated by aligning the feature sequences of source and target speakers implicitly using attention mechanism. At conversion stage, acoustic features and durations of source utterances are converted simultaneously using the unified acoustic model. Mel-scale spectrograms are adopted as acoustic features which contain both excitation and vocal tract descriptions of speech signals. The bottleneck features extracted from source speech using an automatic speech recognition (ASR) model are appended as auxiliary input. A WaveNet vocoder conditioned on Mel-spectrograms is built to reconstruct waveforms from the outputs of the SCENT model. It is worth noting that our proposed method can achieve appropriate duration conversion which is difficult in conventional methods. Experimental results show that our proposed method

Stats

• citations98
• S2 citations—
• github stars0
• HF likes0
• heat score14.97
• arxiv keyzhang2018sequence

Related papers

• Non-parallel Sequence-to-sequence Voice Conversion With Disentangled Linguistic And Speaker Representations (2019)14.02
• Convs2s-vc: Fully Convolutional Sequence-to-sequence Voice Conversion (2018)12.68
• Hierarchical Sequence To Sequence Voice Conversion With Limited Data (2019)0.00
• ACE-VC: Adaptive And Controllable Voice Conversion Using Explicitly Disentangled Self-supervised Speech Representations (2023)0.00
• Svsnet: An End-to-end Speaker Voice Similarity Assessment Model (2021)6.34
• Expressive-vc: Highly Expressive Voice Conversion With Attention Fusion Of Bottleneck And Perturbation Features (2022)9.03
• MAIN-VC: Lightweight Speech Representation Disentanglement For One-shot Voice Conversion (2024)3.58
• Unsupervised Singing Voice Conversion (2019)11.19