Exploring Machine Speech Chain For Domain Adaptation And Few-shot Speaker Adaptation

Abstract

Machine Speech Chain, which integrates both end-to-end (E2E) automatic speech recognition (ASR) and text-to-speech (TTS) into one circle for joint training, has been proven to be effective in data augmentation by leveraging large amounts of unpaired data. In this paper, we explore the TTS->ASR pipeline in speech chain to do domain adaptation for both neural TTS and E2E ASR models, with only text data from target domain. We conduct experiments by adapting from audiobook domain (LibriSpeech) to presentation domain (TED-LIUM), there is a relative word error rate (WER) reduction of 10% for the E2E ASR model on the TED-LIUM test set, and a relative WER reduction of 51.5% in synthetic speech generated by neural TTS in the presentation domain. Further, we apply few-shot speaker adaptation for the E2E ASR by using a few utterances from target speakers in an unsupervised way, results in additional gains.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyue2021exploring

Related papers

• Machine Speech Chain With One-shot Speaker Adaptation (2018)11.29
• A Simple Baseline For Domain Adaptation In End To End ASR Systems Using Synthetic Data (2022)7.16
• TDASS: Target Domain Adaptation Speech Synthesis Framework For Multi-speaker Low-resource TTS (2022)0.00
• Tokenchain: A Discrete Speech Chain Via Semantic Token Modeling (2025)0.00
• Enhancing Synthetic Training Data For Speech Commands: From Asr-based Filtering To Domain Adaptation In SSL Latent Space (2024)0.00
• Listening While Speaking And Visualizing: Improving ASR Through Multimodal Chain (2019)4.52
• Text-only Domain Adaptation Using Unified Speech-text Representation In Transducer (2023)4.52
• Augmenting Images For ASR And TTS Through Single-loop And Dual-loop Multimodal Chain Framework (2020)3.58