Streaming Punctuation: A Novel Punctuation Technique Leveraging Bidirectional Context For Continuous Speech Recognition

Abstract

While speech recognition Word Error Rate (WER) has reached human parity for English, continuous speech recognition scenarios such as voice typing and meeting transcriptions still suffer from segmentation and punctuation problems, resulting from irregular pausing patterns or slow speakers. Transformer sequence tagging models are effective at capturing long bi-directional context, which is crucial for automatic punctuation. Automatic Speech Recognition (ASR) production systems, however, are constrained by real-time requirements, making it hard to incorporate the right context when making punctuation decisions. Context within the segments produced by ASR decoders can be helpful but limiting in overall punctuation performance for a continuous speech session. In this paper, we propose a streaming approach for punctuation or re-punctuation of ASR output using dynamic decoding windows and measure its impact on punctuation and segmentation accuracy across scenarios. The new system tackles over

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keybehre2023streaming

Related papers

• Improved Training For End-to-end Streaming Automatic Speech Recognition Model With Punctuation (2023)0.00
• End To End ASR System With Automatic Punctuation Insertion (2020)0.00
• Improving Punctuation Restoration For Speech Transcripts Via External Data (2021)4.52
• Multimodal Semi-supervised Learning Framework For Punctuation Prediction In Conversational Speech (2020)9.59
• Controllable Time-delay Transformer For Real-time Punctuation Prediction And Disfluency Detection (2020)10.48
• Unified Multimodal Punctuation Restoration Framework For Mixed-modality Corpus (2022)7.16
• A Light-weight And Efficient Punctuation And Word Casing Prediction Model For On-device Streaming ASR (2024)0.00
• Longer Is (not Necessarily) Stronger: Punctuated Long-sequence Training For Enhanced Speech Recognition And Translation (2024)4.52