Very Low Complexity Speech Synthesis Using Framewise Autoregressive GAN (FARGAN) With Pitch Prediction

Abstract

Neural vocoders are now being used in a wide range of speech processing applications. In many of those applications, the vocoder can be the most complex component, so finding lower complexity algorithms can lead to significant practical benefits. In this work, we propose FARGAN, an autoregressive vocoder that takes advantage of long-term pitch prediction to synthesize high-quality speech in small subframes, without the need for teacher-forcing. Experimental results show that the proposed 600~MFLOPS FARGAN vocoder can achieve both higher quality and lower complexity than existing low-complexity vocoders. The quality even matches that of existing higher-complexity vocoders.

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keyvalin2024very

Related papers

• Framewise Wavegan: High Speed Adversarial Vocoder In Time Domain With Very Low Computational Complexity (2022)7.16
• Vocgan: A High-fidelity Real-time Vocoder With A Hierarchically-nested Adversarial Network (2020)12.54
• TFGAN: Time And Frequency Domain Based Generative Adversarial Network For High-fidelity Speech Synthesis (2020)0.00
• Analysis By Adversarial Synthesis -- A Novel Approach For Speech Vocoding (2019)3.58
• Bigvgan: A Universal Neural Vocoder With Large-scale Training (2022)6.17
• Parallel Synthesis For Autoregressive Speech Generation (2022)4.52
• Puffin: Pitch-synchronous Neural Waveform Generation For Fullband Speech On Modest Devices (2022)3.58
• MFCCGAN: A Novel Mfcc-based Speech Synthesizer Using Adversarial Learning (2023)5.84