Controllable Time-delay Transformer For Real-time Punctuation Prediction And Disfluency Detection

Abstract

With the increased applications of automatic speech recognition (ASR) in recent years, it is essential to automatically insert punctuation marks and remove disfluencies in transcripts, to improve the readability of the transcripts as well as the performance of subsequent applications, such as machine translation, dialogue systems, and so forth. In this paper, we propose a Controllable Time-delay Transformer (CT-Transformer) model that jointly completes the punctuation prediction and disfluency detection tasks in real time. The CT-Transformer model facilitates freezing partial outputs with controllable time delay to fulfill the real-time constraints in partial decoding required by subsequent applications. We further propose a fast decoding strategy to minimize latency while maintaining competitive performance. Experimental results on the IWSLT2011 benchmark dataset and an in-house Chinese annotated dataset demonstrate that the proposed approach outperforms the previous state-of-the-art

Stats

• citations24
• S2 citations—
• github stars0
• HF likes0
• heat score10.48
• arxiv keychen2020controllable

Related papers

• Improved Training For End-to-end Streaming Automatic Speech Recognition Model With Punctuation (2023)0.00
• Automatic Spelling Correction With Transformer For Ctc-based End-to-end Speech Recognition (2019)0.00
• Streaming Joint Speech Recognition And Disfluency Detection (2022)0.00
• Streaming Punctuation: A Novel Punctuation Technique Leveraging Bidirectional Context For Continuous Speech Recognition (2023)4.52
• A Light-weight And Efficient Punctuation And Word Casing Prediction Model For On-device Streaming ASR (2024)0.00
• Spike-triggered Non-autoregressive Transformer For End-to-end Speech Recognition (2020)11.39
• Synchronous Transformers For End-to-end Speech Recognition (2019)12.02
• Text-conditioned Transformer For Automatic Pronunciation Error Detection (2020)10.48