Speech Recognition On TV Series With Video-guided Post-asr Correction

Abstract

Automatic Speech Recognition (ASR) has achieved remarkable success with deep learning, driving advancements in conversational artificial intelligence, media transcription, and assistive technologies. However, ASR systems still struggle in complex environments such as TV series, where multiple speakers, overlapping speech, domain-specific terminology, and long-range contextual dependencies pose significant challenges to transcription accuracy. Existing approaches fail to explicitly leverage the rich temporal and contextual information available in the video. To address this limitation, we propose a Video-Guided Post-ASR Correction (VPC) framework that uses a Video-Large Multimodal Model (VLMM) to capture video context and refine ASR outputs. Evaluations on a TV-series benchmark show that our method consistently improves transcription accuracy in complex multimedia environments.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2025speech

Related papers

• Improving Medical Speech-to-text Accuracy With Vision-language Pre-training Model (2023)7.81
• End-to-end Multimodal Speech Recognition (2018)10.21
• Listen, Look And Deliberate: Visual Context-aware Speech Recognition Using Pre-trained Text-video Representations (2020)5.84
• Leveraging Broadcast Media Subtitle Transcripts For Automatic Speech Recognition And Subtitling (2025)2.26
• Learning To Jointly Transcribe And Subtitle For End-to-end Spontaneous Speech Recognition (2022)5.84
• Robust Audio-visual Speech Enhancement: Correcting Misassignments In Complex Environments With Advanced Post-processing (2024)0.00
• VAIS ASR: Building A Conversational Speech Recognition System Using Language Model Combination (2019)0.00
• Listening And Seeing Again: Generative Error Correction For Audio-visual Speech Recognition (2025)1.20