Corpus Synthesis For Zero-shot ASR Domain Adaptation Using Large Language Models

Abstract

While Automatic Speech Recognition (ASR) systems are widely used in many real-world applications, they often do not generalize well to new domains and need to be finetuned on data from these domains. However, target-domain data usually are not readily available in many scenarios. In this paper, we propose a new strategy for adapting ASR models to new target domains without any text or speech from those domains. To accomplish this, we propose a novel data synthesis pipeline that uses a Large Language Model (LLM) to generate a target domain text corpus, and a state-of-the-art controllable speech synthesis model to generate the corresponding speech. We propose a simple yet effective in-context instruction finetuning strategy to increase the effectiveness of LLM in generating text corpora for new domains. Experiments on the SLURP dataset show that the proposed method achieves an average relative word error rate improvement of \(28%\) on unseen target domains without any performance drop in

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keysu2023corpus

Related papers

• Prompting Large Language Models For Zero-shot Domain Adaptation In Speech Recognition (2023)0.00
• Harnessing The Zero-shot Power Of Instruction-tuned Large Language Model In End-to-end Speech Recognition (2023)0.00
• Hard-synth: Synthesizing Diverse Hard Samples For ASR Using Zero-shot TTS And LLM (2024)0.00
• Zero-resource Speech Translation And Recognition With Llms (2024)3.58
• A Domain Adaptation Framework For Speech Recognition Systems With Only Synthetic Data (2025)5.24
• Effective Text Adaptation For Llm-based ASR Through Soft Prompt Fine-tuning (2024)5.84
• Generating Data With Text-to-speech And Large-language Models For Conversational Speech Recognition (2024)6.34
• Enhancing Code-switched Text-to-speech Synthesis Capability In Large Language Models With Only Monolingual Corpora (2024)0.00