A Domain Adaptation Framework For Speech Recognition Systems With Only Synthetic Data

Abstract

We introduce DAS (Domain Adaptation with Synthetic data), a novel domain adaptation framework for pre-trained ASR model, designed to efficiently adapt to various language-defined domains without requiring any real data. In particular, DAS first prompts large language models (LLMs) to generate domain-specific texts before converting these texts to speech via text-to-speech technology. The synthetic data is used to fine-tune Whisper with Low-Rank Adapters (LoRAs) for targeted domains such as music, weather, and sports. We introduce a novel one-pass decoding strategy that merges predictions from multiple LoRA adapters efficiently during the auto-regressive text generation process. Experimental results show significant improvements, reducing the Word Error Rate (WER) by 10% to 17% across all target domains compared to the original model, with minimal performance regression in out-of-domain settings (e.g., -1% on Librispeech test sets). We also demonstrate that DAS operates efficiently duri

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keytran2025a

Related papers

• A Simple Baseline For Domain Adaptation In End To End ASR Systems Using Synthetic Data (2022)7.16
• Corpus Synthesis For Zero-shot ASR Domain Adaptation Using Large Language Models (2023)5.84
• Training Data Augmentation For Dysarthric Automatic Speech Recognition By Text-to-dysarthric-speech Synthesis (2024)10.48
• Enhancing Synthetic Training Data For Speech Commands: From Asr-based Filtering To Domain Adaptation In SSL Latent Space (2024)0.00
• Generating Data With Text-to-speech And Large-language Models For Conversational Speech Recognition (2024)6.34
• Domain Adaptation Of Low-resource Target-domain Models Using Well-trained ASR Conformer Models (2022)4.52
• Unsupervised Domain Adaptation For Speech Recognition With Unsupervised Error Correction (2022)5.24
• MADI: Inter-domain Matching And Intra-domain Discrimination For Cross-domain Speech Recognition (2023)7.50