Speechsplit 2.0: Unsupervised Speech Disentanglement For Voice Conversion Without Tuning Autoencoder Bottlenecks

Abstract

SpeechSplit can perform aspect-specific voice conversion by disentangling speech into content, rhythm, pitch, and timbre using multiple autoencoders in an unsupervised manner. However, SpeechSplit requires careful tuning of the autoencoder bottlenecks, which can be time-consuming and less robust. This paper proposes SpeechSplit 2.0, which constrains the information flow of the speech component to be disentangled on the autoencoder input using efficient signal processing methods instead of bottleneck tuning. Evaluation results show that SpeechSplit 2.0 achieves comparable performance to SpeechSplit in speech disentanglement and superior robustness to the bottleneck size variations. Our code is available at https://github.com/biggytruck/SpeechSplit2.

Stats

• citations0
• S2 citations—
• github stars136
• HF likes0
• heat score4.27
• arxiv keychan2022speechsplit

Code

• biggytruck/SpeechSplit2—

Related papers

• Unsupervised Speech Decomposition Via Triple Information Bottleneck (2020)5.69
• Tokensplit: Using Discrete Speech Representations For Direct, Refined, And Transcript-conditioned Speech Separation And Recognition (2023)7.50
• EAD-VC: Enhancing Speech Auto-disentanglement For Voice Conversion With IFUB Estimator And Joint Text-guided Consistent Learning (2024)4.52
• Many-to-many Voice Conversion Based Feature Disentanglement Using Variational Autoencoder (2021)7.81
• Automatic Speech Disentanglement For Voice Conversion Using Rank Module And Speech Augmentation (2023)4.52
• Vasab: The Variable Size Adaptive Information Bottleneck For Disentanglement On Speech And Singing Voice (2023)0.00
• Contrastive Predictive Coding Supported Factorized Variational Autoencoder For Unsupervised Learning Of Disentangled Speech Representations (2020)8.09
• ACE-VC: Adaptive And Controllable Voice Conversion Using Explicitly Disentangled Self-supervised Speech Representations (2023)0.00