Enhancing The Unified Streaming And Non-streaming Model With Contrastive Learning
2023 · Yuting Yang, Yuke Li, Binbin Du
Abstract
The unified streaming and non-streaming speech recognition model has achieved great success due to its comprehensive capabilities. In this paper, we propose to improve the accuracy of the unified model by bridging the inherent representation gap between the streaming and non-streaming modes with a contrastive objective. Specifically, the top-layer hidden representation at the same frame of the streaming and non-streaming modes are regarded as a positive pair, encouraging the representation of the streaming mode close to its non-streaming counterpart. The multiple negative samples are randomly selected from the rest frames of the same sample under the non-streaming mode. Experimental results demonstrate that the proposed method achieves consistent improvements toward the unified model in both streaming and non-streaming modes. Our method achieves CER of 4.66% in the streaming mode and CER of 4.31% in the non-streaming mode, which sets a new state-of-the-art on the AISHELL-1 benchmark.
Authors
(none)
Tags
Stats
Related papers
- Unified Streaming And Non-streaming Two-pass End-to-end Model For Speech Recognition (2020)0.00
- U2++: Unified Two-pass Bidirectional End-to-end Model For Speech Recognition (2021)0.00
- Cascaded Encoders For Unifying Streaming And Non-streaming ASR (2020)12.47
- Dynamic Chunk Convolution For Unified Streaming And Non-streaming Conformer ASR (2023)6.77
- Bridging The Gap Between Streaming And Non-streaming ASR Systems Bydistilling Ensembles Of CTC And RNN-T Models (2021)3.58
- Joint Optimization Of Streaming And Non-streaming Automatic Speech Recognition With Multi-decoder And Knowledge Distillation (2024)0.00
- Two Stage Contextual Word Filtering For Context Bias In Unified Streaming And Non-streaming Transducer (2023)7.16
- Transformer Transducer: One Model Unifying Streaming And Non-streaming Speech Recognition (2020)0.00