Dynamic Latency Speech Recognition With Asynchronous Revision

Abstract

In this work we propose an inference technique, asynchronous revision, to unify streaming and non-streaming speech recognition models. Specifically, we achieve dynamic latency with only one model by using arbitrary right context during inference. The model is composed of a stack of convolutional layers for audio encoding. In inference stage, the history states of encoder and decoder can be asynchronously revised to trade off between the latency and the accuracy of the model. To alleviate training and inference mismatch, we propose a training technique, segment cropping, which randomly splits input utterances into several segments with forward connections. This allows us to have dynamic latency speech recognition results with large improvements in accuracy. Experiments show that our dynamic latency model with asynchronous revision gives 8%-14% relative improvements over the streaming models.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhuang2020dynamic

Related papers

• Dynamic Latency For Ctc-based Streaming Automatic Speech Recognition With Emformer (2022)0.00
• Dynamic Chunk Convolution For Unified Streaming And Non-streaming Conformer ASR (2023)6.77
• Dctx-conformer: Dynamic Context Carry-over For Low Latency Unified Streaming And Non-streaming Conformer ASR (2023)2.26
• Iterative Autoregression: A Novel Trick To Improve Your Low-latency Speech Enhancement Model (2022)5.24
• Improving Streaming Speech Recognition With Time-shifted Contextual Attention And Dynamic Right Context Masking (2025)2.26
• Dualvc 3: Leveraging Language Model Generated Pseudo Context For End-to-end Low Latency Streaming Voice Conversion (2024)0.00
• Latency-controlled Neural Architecture Search For Streaming Speech Recognition (2021)0.00
• Stateful Conformer With Cache-based Inference For Streaming Automatic Speech Recognition (2023)8.60