Fast-u2++: Fast And Accurate End-to-end Speech Recognition In Joint Ctc/attention Frames

Abstract

Recently, the unified streaming and non-streaming two-pass (U2/U2++) end-to-end model for speech recognition has shown great performance in terms of streaming capability, accuracy and latency. In this paper, we present fast-U2++, an enhanced version of U2++ to further reduce partial latency. The core idea of fast-U2++ is to output partial results of the bottom layers in its encoder with a small chunk, while using a large chunk in the top layers of its encoder to compensate the performance degradation caused by the small chunk. Moreover, we use knowledge distillation method to reduce the token emission latency. We present extensive experiments on Aishell-1 dataset. Experiments and ablation studies show that compared to U2++, fast-U2++ reduces model latency from 320ms to 80ms, and achieves a character error rate (CER) of 5.06% with a streaming setup.

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyliang2022fast

Related papers

• U2++: Unified Two-pass Bidirectional End-to-end Model For Speech Recognition (2021)0.00
• Unified Streaming And Non-streaming Two-pass End-to-end Model For Speech Recognition (2020)0.00
• Unified End-to-end Speech Recognition And Endpointing For Fast And Efficient Speech Systems (2022)5.24
• Two-pass End-to-end Speech Recognition (2019)13.97
• Two-stage Augmentation And Adaptive CTC Fusion For Improved Robustness Of Multi-stream End-to-end ASR (2021)2.26
• FPETS : Fully Parallel End-to-end Text-to-speech System (2018)4.52
• Unity: Two-pass Direct Speech-to-speech Translation With Discrete Units (2022)9.59
• Streaming Attention-based Models With Augmented Memory For End-to-end Speech Recognition (2020)5.84