Gansynth: Adversarial Neural Audio Synthesis

Abstract

Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence. Autoregressive models, such as WaveNet, model local structure at the expense of global latent structure and slow iterative sampling, while Generative Adversarial Networks (GANs), have global latent conditioning and efficient parallel sampling, but struggle to generate locally-coherent audio waveforms. Herein, we demonstrate that GANs can in fact generate high-fidelity and locally-coherent audio by modeling log magnitudes and instantaneous frequencies with sufficient frequency resolution in the spectral domain. Through extensive empirical investigations on the NSynth dataset, we demonstrate that GANs are able to outperform strong WaveNet baselines on automated and human evaluation metrics, and efficiently generate audio several orders of magnitude faster than their autoregressive counterparts.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyengel2019gansynth

Related papers

• Adversarial Audio Synthesis (2018)0.00
• High Fidelity Speech Synthesis With Adversarial Networks (2019)0.00
• Waveform Generation For Text-to-speech Synthesis Using Pitch-synchronous Multi-scale Generative Adversarial Networks (2018)8.35
• Hifi-gan: Generative Adversarial Networks For Efficient And High Fidelity Speech Synthesis (2020)0.00
• Melgan: Generative Adversarial Networks For Conditional Waveform Synthesis (2019)0.00
• Adversarial Generation Of Time-frequency Features With Application In Audio Synthesis (2019)0.00
• Analysis By Adversarial Synthesis -- A Novel Approach For Speech Vocoding (2019)3.58
• Bandwidth Extension On Raw Audio Via Generative Adversarial Networks (2019)0.00