Investigation Of F0 Conditioning And Fully Convolutional Networks In Variational Autoencoder Based Voice Conversion

Abstract

In this work, we investigate the effectiveness of two techniques for improving variational autoencoder (VAE) based voice conversion (VC). First, we reconsider the relationship between vocoder features extracted using the high quality vocoders adopted in conventional VC systems, and hypothesize that the spectral features are in fact F0 dependent. Such hypothesis implies that during the conversion phase, the latent codes and the converted features in VAE based VC are in fact source F0 dependent. To this end, we propose to utilize the F0 as an additional input of the decoder. The model can learn to disentangle the latent code from the F0 and thus generates converted F0 dependent converted features. Second, to better capture temporal dependencies of the spectral features and the F0 pattern, we replace the frame wise conversion structure in the original VAE based VC framework with a fully convolutional network structure. Our experiments demonstrate that the degree of disentanglement as well

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhuang2019investigation

Related papers

• F0-consistent Many-to-many Non-parallel Voice Conversion Via Conditional Autoencoder (2020)13.17
• ACVAE-VC: Non-parallel Many-to-many Voice Conversion With Auxiliary Classifier Variational Autoencoder (2018)14.69
• Refined Wavenet Vocoder For Variational Autoencoder Based Voice Conversion (2018)7.50
• Conditional Deep Hierarchical Variational Autoencoder For Voice Conversion (2021)0.00
• Many-to-many Voice Conversion Based Feature Disentanglement Using Variational Autoencoder (2021)7.81
• Non-parallel Voice Conversion With Cyclic Variational Autoencoder (2019)12.10
• Improved Prosody From Learned F0 Codebook Representations For VQ-VAE Speech Waveform Reconstruction (2020)7.50
• Voice Reenactment With F0 And Timing Constraints And Adversarial Learning Of Conversions (2021)2.26