Benchmarking Japanese Speech Recognition On ASR-LLM Setups With Multi-pass Augmented Generative Error Correction

Abstract

With the strong representational power of large language models (LLMs), generative error correction (GER) for automatic speech recognition (ASR) aims to provide semantic and phonetic refinements to address ASR errors. This work explores how LLM-based GER can enhance and expand the capabilities of Japanese language processing, presenting the first GER benchmark for Japanese ASR with 0.9-2.6k text utterances. We also introduce a new multi-pass augmented generative error correction (MPA GER) by integrating multiple system hypotheses on the input side with corrections from multiple LLMs on the output side and then merging them. To the best of our knowledge, this is the first investigation of the use of LLMs for Japanese GER, which involves second-pass language modeling on the output transcriptions generated by the ASR system (e.g., N-best hypotheses). Our experiments demonstrated performance improvement in the proposed methods of ASR quality and generalization both in SPREDS-U1-ja and CSJ

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyko2024benchmarking

Related papers

• Multi-stage Large Language Model Correction For Speech Recognition (2023)0.00
• Listening And Seeing Again: Generative Error Correction For Audio-visual Speech Recognition (2025)1.20
• Large Language Model Based Generative Error Correction: A Challenge And Baselines For Speech Recognition, Speaker Tagging, And Emotion Recognition (2024)7.81
• A Comparative Study On Neural Architectures And Training Methods For Japanese Speech Recognition (2021)7.50
• La-rag:enhancing Llm-based ASR Accuracy With Retrieval-augmented Generation (2024)0.00
• Generative Error Correction For Code-switching Speech Recognition Using Large Language Models (2023)0.00
• ASR Error Correction Using Large Language Models (2024)9.41
• Lipger: Visually-conditioned Generative Error Correction For Robust Automatic Speech Recognition (2024)2.26