Bifocal Neural ASR: Exploiting Keyword Spotting For Inference Optimization

Abstract

We present Bifocal RNN-T, a new variant of the Recurrent Neural Network Transducer (RNN-T) architecture designed for improved inference time latency on speech recognition tasks. The architecture enables a dynamic pivot for its runtime compute pathway, namely taking advantage of keyword spotting to select which component of the network to execute for a given audio frame. To accomplish this, we leverage a recurrent cell we call the Bifocal LSTM (BFLSTM), which we detail in the paper. The architecture is compatible with other optimization strategies such as quantization, sparsification, and applying time-reduction layers, making it especially applicable for deployed, real-time speech recognition settings. We present the architecture and report comparative experimental results on voice-assistant speech recognition tasks. Specifically, we show our proposed Bifocal RNN-T can improve inference cost by 29.1% with matching word error rates and only a minor increase in memory size.

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keymacoskey2021bifocal

Related papers

• Dual-attention Neural Transducers For Efficient Wake Word Spotting In Speech Recognition (2023)5.24
• Improved Neural Language Model Fusion For Streaming Recurrent Neural Network Transducer (2020)8.82
• CIF-T: A Novel Cif-based Transducer Architecture For Automatic Speech Recognition (2023)0.00
• RNN-T For Latency Controlled ASR With Improved Beam Search (2019)0.00
• Streaming Small-footprint Keyword Spotting Using Sequence-to-sequence Models (2017)12.40
• Streaming Multi-speaker ASR With RNN-T (2020)10.07
• Improving RNN Transducer Based ASR With Auxiliary Tasks (2020)9.59
• Efficient Keyword Spotting Using Time Delay Neural Networks (2018)10.21