Latency-controlled Neural Architecture Search For Streaming Speech Recognition

Abstract

Neural architecture search (NAS) has attracted much attention and has been explored for automatic speech recognition (ASR). In this work, we focus on streaming ASR scenarios and propose the latency-controlled NAS for acoustic modeling. First, based on the vanilla neural architecture, normal cells are altered to causal cells to control the total latency of the architecture. Second, a revised operation space with a smaller receptive field is proposed to generate the final architecture with low latency. Extensive experiments show that: 1) Based on the proposed neural architecture, the neural networks with a medium latency of 550ms (millisecond) and a low latency of 190ms can be learned in the vanilla and revised operation space respectively. 2) For the low latency setting, the evaluation network can achieve more than 19% (average on the four test sets) relative improvements compared with the hybrid CLDNN baseline, on a 10k-hour large-scale dataset.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhe2021latency

Related papers

• Improved Conformer-based End-to-end Speech Recognition Using Neural Architecture Search (2021)0.00
• Efficient Neural Architecture Search For End-to-end Speech Recognition Via Straight-through Gradients (2020)8.35
• Minimum Latency Training Strategies For Streaming Sequence-to-sequence ASR (2020)10.07
• Lookahead When It Matters: Adaptive Non-causal Transformers For Streaming Neural Transducers (2023)0.00
• RNN-T For Latency Controlled ASR With Improved Beam Search (2019)0.00
• Streaming Audio-visual Speech Recognition With Alignment Regularization (2022)3.58
• Mask-ctc-based Encoder Pre-training For Streaming End-to-end Speech Recognition (2023)0.00
• Streaming Parallel Transducer Beam Search With Fast-slow Cascaded Encoders (2022)0.00