F0-consistent Many-to-many Non-parallel Voice Conversion Via Conditional Autoencoder

Abstract

Non-parallel many-to-many voice conversion remains an interesting but challenging speech processing task. Many style-transfer-inspired methods such as generative adversarial networks (GANs) and variational autoencoders (VAEs) have been proposed. Recently, AutoVC, a conditional autoencoders (CAEs) based method achieved state-of-the-art results by disentangling the speaker identity and speech content using information-constraining bottlenecks, and it achieves zero-shot conversion by swapping in a different speaker's identity embedding to synthesize a new voice. However, we found that while speaker identity is disentangled from speech content, a significant amount of prosodic information, such as source F0, leaks through the bottleneck, causing target F0 to fluctuate unnaturally. Furthermore, AutoVC has no control of the converted F0 and thus unsuitable for many applications. In the paper, we modified and improved autoencoder-based voice conversion to disentangle content, F0, and speaker

Stats

• citations56
• S2 citations—
• github stars0
• HF likes0
• heat score13.17
• arxiv keyqian2020f0

Related papers

• Investigation Of F0 Conditioning And Fully Convolutional Networks In Variational Autoencoder Based Voice Conversion (2019)0.00
• Many-to-many Voice Conversion Based Feature Disentanglement Using Variational Autoencoder (2021)7.81
• ACVAE-VC: Non-parallel Many-to-many Voice Conversion With Auxiliary Classifier Variational Autoencoder (2018)14.69
• AUTOVC: Zero-shot Voice Style Transfer With Only Autoencoder Loss (2019)0.00
• Fastvc: Fast Voice Conversion With Non-parallel Data (2020)5.24
• Voice Conversion With Diverse Intonation Using Conditional Variational Auto-encoder (2025)0.00
• Many-to-many Voice Conversion Using Cycle-consistent Variational Autoencoder With Multiple Decoders (2019)6.34
• Non-parallel Voice Conversion With Cyclic Variational Autoencoder (2019)12.10