On The Problem Of Text-to-speech Model Selection For Synthetic Data Generation In Automatic Speech Recognition

Abstract

The rapid development of neural text-to-speech (TTS) systems enabled its usage in other areas of natural language processing such as automatic speech recognition (ASR) or spoken language translation (SLT). Due to the large number of different TTS architectures and their extensions, selecting which TTS systems to use for synthetic data creation is not an easy task. We use the comparison of five different TTS decoder architectures in the scope of synthetic data generation to show the impact on CTC-based speech recognition training. We compare the recognition results to computable metrics like NISQA MOS and intelligibility, finding that there are no clear relations to the ASR performance. We also observe that for data generation auto-regressive decoding performs better than non-autoregressive decoding, and propose an approach to quantify TTS generalization capabilities.

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyrossenbach2024on

Related papers

• On The Effect Of Purely Synthetic Training Data For Different Automatic Speech Recognition Architectures (2024)0.00
• Comparing The Benefit Of Synthetic Training Data For Various Automatic Speech Recognition Architectures (2021)5.24
• Generating Synthetic Audio Data For Attention-based Speech Recognition Systems (2019)12.68
• On The Relevance Of Phoneme Duration Variability Of Synthesized Training Data For Automatic Speech Recognition (2023)2.26
• Evaluating Text-to-speech Synthesis From A Large Discrete Token-based Speech Language Model (2024)0.00
• Tts-by-tts 2: Data-selective Augmentation For Neural Speech Synthesis Using Ranking Support Vector Machine With Variational Autoencoder (2022)4.52
• Tts-by-tts: Tts-driven Data Augmentation For Fast And High-quality Speech Synthesis (2020)9.59
• Generating Data With Text-to-speech And Large-language Models For Conversational Speech Recognition (2024)6.34