La-rag:enhancing Llm-based ASR Accuracy With Retrieval-augmented Generation

Abstract

Recent advancements in integrating speech information into large language models (LLMs) have significantly improved automatic speech recognition (ASR) accuracy. However, existing methods often constrained by the capabilities of the speech encoders under varied acoustic conditions, such as accents. To address this, we propose LA-RAG, a novel Retrieval-Augmented Generation (RAG) paradigm for LLM-based ASR. LA-RAG leverages fine-grained token-level speech datastores and a speech-to-speech retrieval mechanism to enhance ASR accuracy via LLM in-context learning (ICL) capabilities. Experiments on Mandarin and various Chinese dialect datasets demonstrate significant improvements in ASR accuracy compared to existing methods, validating the effectiveness of our approach, especially in handling accent variations.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyli2024la

Related papers

• Audiorag+: Feedback-driven Retrieval-augmented Audio Generation With Large Audio Language Models (2025)0.00
• GEC-RAG: Improving Generative Error Correction Via Retrieval-augmented Generation For Automatic Speech Recognition Systems (2025)0.00
• Wavrag: Audio-integrated Retrieval Augmented Generation For Spoken Dialogue Models (2025)5.24
• Effective Text Adaptation For Llm-based ASR Through Soft Prompt Fine-tuning (2024)5.84
• Retrieval Augmented Generation In Prompt-based Text-to-speech Synthesis With Context-aware Contrastive Language-audio Pretraining (2024)0.00
• Lipger: Visually-conditioned Generative Error Correction For Robust Automatic Speech Recognition (2024)2.26
• It's Never Too Late: Fusing Acoustic Information Into Large Language Models For Automatic Speech Recognition (2024)0.00
• Memory Augmented Lookup Dictionary Based Language Modeling For Automatic Speech Recognition (2022)0.00