Adversarial Feature Learning And Unsupervised Clustering Based Speech Synthesis For Found Data With Acoustic And Textual Noise

Abstract

Attention-based sequence-to-sequence (seq2seq) speech synthesis has achieved extraordinary performance. But a studio-quality corpus with manual transcription is necessary to train such seq2seq systems. In this paper, we propose an approach to build high-quality and stable seq2seq based speech synthesis system using challenging found data, where training speech contains noisy interferences (acoustic noise) and texts are imperfect speech recognition transcripts (textual noise). To deal with text-side noise, we propose a VQVAE based heuristic method to compensate erroneous linguistic feature with phonetic information learned directly from speech. As for the speech-side noise, we propose to learn a noise-independent feature in the auto-regressive decoder through adversarial training and data augmentation, which does not need an extra speech enhancement model. Experiments show the effectiveness of the proposed approach in dealing with text-side and speech-side noise. Surpassing the denoisin

Stats

• citations8
• S2 citations—
• github stars0
• HF likes0
• heat score7.16
• arxiv keyyang2020adversarial

Related papers

• Unpaired Speech Enhancement By Acoustic And Adversarial Supervision For Speech Recognition (2018)10.21
• Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languages (2020)9.59
• Adversarial Data Augmentation Using VAE-GAN For Disordered Speech Recognition (2022)0.00
• Incorporating Real-world Noisy Speech In Neural-network-based Speech Enhancement Systems (2021)5.84
• Data Efficient Voice Cloning From Noisy Samples With Domain Adversarial Training (2020)9.92
• Improving Accented Speech Recognition Using Data Augmentation Based On Unsupervised Text-to-speech Synthesis (2024)0.00
• Tts-by-tts 2: Data-selective Augmentation For Neural Speech Synthesis Using Ranking Support Vector Machine With Variational Autoencoder (2022)4.52
• Disentangling Speech And Non-speech Components For Building Robust Acoustic Models From Found Data (2019)0.00