Quickvc: Any-to-many Voice Conversion Using Inverse Short-time Fourier Transform For Faster Conversion

Abstract

With the development of automatic speech recognition (ASR) and text-to-speech (TTS) technology, high-quality voice conversion (VC) can be achieved by extracting source content information and target speaker information to reconstruct waveforms. However, current methods still require improvement in terms of inference speed. In this study, we propose a lightweight VITS-based VC model that uses the HuBERT-Soft model to extract content information features without speaker information. Through subjective and objective experiments on synthesized speech, the proposed model demonstrates competitive results in terms of naturalness and similarity. Importantly, unlike the original VITS model, we use the inverse short-time Fourier transform (iSTFT) to replace the most computationally expensive part. Experimental results show that our model can generate samples at over 5000 kHz on the 3090 GPU and over 250 kHz on the i9-10900K CPU, achieving competitive speed for the same hardware configuration.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2023quickvc

Related papers

• Fastvc: Fast Voice Conversion With Non-parallel Data (2020)5.24
• AVQVC: One-shot Voice Conversion By Vector Quantization With Applying Contrastive Learning (2022)12.40
• Lightweight And High-fidelity End-to-end Text-to-speech With Multi-band Generation And Inverse Short-time Fourier Transform (2022)14.57
• Voice Filter: Few-shot Text-to-speech Speaker Adaptation Using Voice Conversion As A Post-processing Module (2022)8.35
• VQVC+: One-shot Voice Conversion By Vector Quantization And U-net Architecture (2020)13.34
• VCVTS: Multi-speaker Video-to-speech Synthesis Via Cross-modal Knowledge Transfer From Voice Conversion (2022)6.77
• AC-VC: Non-parallel Low Latency Phonetic Posteriorgrams Based Voice Conversion (2021)7.50
• Efficient Non-autoregressive GAN Voice Conversion Using Vqwav2vec Features And Dynamic Convolution (2022)0.00