Enhancing Gan-based Vocoders With Contrastive Learning Under Data-limited Condition

Abstract

Vocoder models have recently achieved substantial progress in generating authentic audio comparable to human quality while significantly reducing memory requirement and inference time. However, these data-hungry generative models require large-scale audio data for learning good representations. In this paper, we apply contrastive learning methods in training the vocoder to improve the perceptual quality of the vocoder without modifying its architecture or adding more data. We design an auxiliary task with mel-spectrogram contrastive learning to enhance the utterance-level quality of the vocoder model under data-limited conditions. We also extend the task to include waveforms to improve the multi-modality comprehension of the model and address the discriminator overfitting problem. We optimize the additional task simultaneously with GAN training objectives. Our results show that the tasks improve model performance substantially in data-limited settings.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyguo2023enhancing

Related papers

• Training Generative Adversarial Network-based Vocoder With Limited Data Using Augmentation-conditional Discriminator (2024)2.26
• Vocgan: A High-fidelity Real-time Vocoder With A Hierarchically-nested Adversarial Network (2020)12.54
• Bigvgan: A Universal Neural Vocoder With Large-scale Training (2022)6.17
• CVC: Contrastive Learning For Non-parallel Voice Conversion (2020)7.50
• VQCPC-GAN: Variable-length Adversarial Audio Synthesis Using Vector-quantized Contrastive Predictive Coding (2021)5.84
• Bemaganv2: Discriminator Combination Strategies For Gan-based Vocoders In Long-term Audio Generation (2025)2.68
• Vnet: A Gan-based Multi-tier Discriminator Network For Speech Synthesis Vocoders (2024)2.26
• Expediting TTS Synthesis With Adversarial Vocoding (2019)6.77