Failing Forward: Improving Generative Error Correction For ASR With Synthetic Data And Retrieval Augmentation

Abstract

Generative Error Correction (GEC) has emerged as a powerful post-processing method to enhance the performance of Automatic Speech Recognition (ASR) systems. However, we show that GEC models struggle to generalize beyond the specific types of errors encountered during training, limiting their ability to correct new, unseen errors at test time, particularly in out-of-domain (OOD) scenarios. This phenomenon amplifies with named entities (NEs), where, in addition to insufficient contextual information or knowledge about the NEs, novel NEs keep emerging. To address these issues, we propose DARAG (Data- and Retrieval-Augmented Generative Error Correction), a novel approach designed to improve GEC for ASR in in-domain (ID) and OOD scenarios. We augment the GEC training dataset with synthetic data generated by prompting LLMs and text-to-speech models, thereby simulating additional errors from which the model can learn. For OOD scenarios, we simulate test-time errors from new domains similarly

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyghosh2024failing

Related papers

• GEC-RAG: Improving Generative Error Correction Via Retrieval-augmented Generation For Automatic Speech Recognition Systems (2025)0.00
• Large Language Model Based Generative Error Correction: A Challenge And Baselines For Speech Recognition, Speaker Tagging, And Emotion Recognition (2024)7.81
• Channel-aware Domain-adaptive Generative Adversarial Network For Robust Speech Recognition (2024)4.52
• Lipger: Visually-conditioned Generative Error Correction For Robust Automatic Speech Recognition (2024)2.26
• ASR Error Correction Using Large Language Models (2024)9.41
• Investigating Generative Adversarial Networks Based Speech Dereverberation For Robust Speech Recognition (2018)10.74
• La-rag:enhancing Llm-based ASR Accuracy With Retrieval-augmented Generation (2024)0.00
• Fine-tuning Of Pre-trained End-to-end Speech Recognition With Generative Adversarial Networks (2021)5.84