Prompting Large Language Models For Zero-shot Domain Adaptation In Speech Recognition

Abstract

The integration of Language Models (LMs) has proven to be an effective way to address domain shifts in speech recognition. However, these approaches usually require a significant amount of target domain text data for the training of LMs. Different from these methods, in this work, with only a domain-specific text prompt, we propose two zero-shot ASR domain adaptation methods using LLaMA, a 7-billion-parameter large language model (LLM). LLM is used in two ways: 1) second-pass rescoring: reranking N-best hypotheses of a given ASR system with LLaMA; 2) deep LLM-fusion: incorporating LLM into the decoder of an encoder-decoder based ASR system. Experiments show that, with only one domain prompt, both methods can effectively reduce word error rates (WER) on out-of-domain TedLium-2 and SPGISpeech datasets. Especially, the deep LLM-fusion has the advantage of better recall of entity and out-of-vocabulary words.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2023prompting

Related papers

• Corpus Synthesis For Zero-shot ASR Domain Adaptation Using Large Language Models (2023)5.84
• Effective Text Adaptation For Llm-based ASR Through Soft Prompt Fine-tuning (2024)5.84
• Zero-resource Speech Translation And Recognition With Llms (2024)3.58
• Harnessing The Zero-shot Power Of Instruction-tuned Large Language Model In End-to-end Speech Recognition (2023)0.00
• Multi-stage Large Language Model Correction For Speech Recognition (2023)0.00
• Wav2prompt: End-to-end Speech Prompt Generation And Tuning For LLM In Zero And Few-shot Learning (2024)0.00
• Chain-of-thought Prompting For Speech Translation (2024)6.34
• On Decoder-only Architecture For Speech-to-text And Large Language Model Integration (2023)0.00