High Performance Sequence-to-sequence Model For Streaming Speech Recognition

Abstract

Recently sequence-to-sequence models have started to achieve state-of-the-art performance on standard speech recognition tasks when processing audio data in batch mode, i.e., the complete audio data is available when starting processing. However, when it comes to performing run-on recognition on an input stream of audio data while producing recognition results in real-time and with low word-based latency, these models face several challenges. For many techniques, the whole audio sequence to be decoded needs to be available at the start of the processing, e.g., for the attention mechanism or the bidirectional LSTM (BLSTM). In this paper, we propose several techniques to mitigate these problems. We introduce an additional loss function controlling the uncertainty of the attention mechanism, a modified beam search identifying partial, stable hypotheses, ways of working with BLSTM in the encoder, and the use of chunked BLSTM. Our experiments show that with the right combination of these te

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keynguyen2020high

Related papers

• Minimum Latency Training Strategies For Streaming Sequence-to-sequence ASR (2020)10.07
• Minimum Latency Training Of Sequence Transducers For Streaming End-to-end Speech Recognition (2022)0.00
• State-of-the-art Speech Recognition With Sequence-to-sequence Models (2017)21.01
• Streaming Attention-based Models With Augmented Memory For End-to-end Speech Recognition (2020)5.84
• An Online Sequence-to-sequence Model For Noisy Speech Recognition (2017)0.00
• Streaming Decoder-only Automatic Speech Recognition With Discrete Speech Units: A Pilot Study (2024)4.52
• Towards Better Decoding And Language Model Integration In Sequence To Sequence Models (2016)15.67
• Chunked Attention-based Encoder-decoder Model For Streaming Speech Recognition (2023)7.81