VQVAE Unsupervised Unit Discovery And Multi-scale Code2spec Inverter For Zerospeech Challenge 2019

Abstract

We describe our submitted system for the ZeroSpeech Challenge 2019. The current challenge theme addresses the difficulty of constructing a speech synthesizer without any text or phonetic labels and requires a system that can (1) discover subword units in an unsupervised way, and (2) synthesize the speech with a target speaker's voice. Moreover, the system should also balance the discrimination score ABX, the bit-rate compression rate, and the naturalness and the intelligibility of the constructed voice. To tackle these problems and achieve the best trade-off, we utilize a vector quantized variational autoencoder (VQ-VAE) and a multi-scale codebook-to-spectrogram (Code2Spec) inverter trained by mean square error and adversarial loss. The VQ-VAE extracts the speech to a latent space, forces itself to map it into the nearest codebook and produces compressed representation. Next, the inverter generates a magnitude spectrogram to the target voice, given the codebook vectors from VQ-VAE. In

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytjandra2019vqvae

Related papers

• Transformer VQ-VAE For Unsupervised Unit Discovery And Speech Synthesis: Zerospeech 2020 Challenge (2020)9.41
• Unsupervised Acoustic Unit Discovery For Speech Synthesis Using Discrete Latent-variable Neural Networks (2019)9.59
• Vector-quantized Neural Networks For Acoustic Unit Discovery In The Zerospeech 2020 Challenge (2020)13.50
• The Neteasegames System For Voice Conversion Challenge 2020 With Vector-quantization Variational Autoencoder And Wavenet (2020)0.00
• Unsupervised Acoustic Unit Representation Learning For Voice Conversion Using Wavenet Auto-encoders (2020)7.16
• The Zero Resource Speech Challenge 2020: Discovering Discrete Subword And Word Units (2020)11.58
• Delightfultts 2: End-to-end Speech Synthesis With Adversarial Vector-quantized Auto-encoders (2022)9.23
• Robust Disentangled Variational Speech Representation Learning For Zero-shot Voice Conversion (2022)10.97