Multi-stage Large Language Model Correction For Speech Recognition

Abstract

In this paper, we investigate the usage of large language models (LLMs) to improve the performance of competitive speech recognition systems. Different from previous LLM-based ASR error correction methods, we propose a novel multi-stage approach that utilizes uncertainty estimation of ASR outputs and reasoning capability of LLMs. Specifically, the proposed approach has two stages: the first stage is about ASR uncertainty estimation and exploits N-best list hypotheses to identify less reliable transcriptions; The second stage works on these identified transcriptions and performs LLM-based corrections. This correction task is formulated as a multi-step rule-based LLM reasoning process, which uses explicitly written rules in prompts to decompose the task into concrete reasoning steps. Our experimental results demonstrate the effectiveness of the proposed method by showing 10% ~ 20% relative improvement in WER over competitive ASR systems -- across multiple test domains and in zero-shot se

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypu2023multi

Related papers

• ASR Error Correction Using Large Language Models (2024)9.41
• Chain Of Correction For Full-text Speech Recognition With Large Language Models (2025)0.00
• Towards Interfacing Large Language Models With ASR Systems Using Confidence Measures And Prompting (2024)7.16
• Harnessing The Zero-shot Power Of Instruction-tuned Large Language Model In End-to-end Speech Recognition (2023)0.00
• It's Never Too Late: Fusing Acoustic Information Into Large Language Models For Automatic Speech Recognition (2024)0.00
• Large Language Model Can Transcribe Speech In Multi-talker Scenarios With Versatile Instructions (2024)11.23
• Generative Error Correction For Code-switching Speech Recognition Using Large Language Models (2023)0.00
• Full-text Error Correction For Chinese Speech Recognition With Large Language Model (2024)4.52