Improving Vision-inspired Keyword Spotting Using Dynamic Module Skipping In Streaming Conformer Encoder

Abstract

Using a vision-inspired keyword spotting framework, we propose an architecture with input-dependent dynamic depth capable of processing streaming audio. Specifically, we extend a conformer encoder with trainable binary gates that allow us to dynamically skip network modules according to the input audio. Our approach improves detection and localization accuracy on continuous speech using Librispeech top-1000 most frequent words while maintaining a small memory footprint. The inclusion of gates also reduces the average amount of processing without affecting the overall performance. These benefits are shown to be even more pronounced using the Google speech commands dataset placed over background noise where up to 97% of the processing is skipped on non-speech inputs, therefore making our method particularly interesting for an always-on keyword spotter.

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keybittar2023improving

Related papers

• End-to-end Streaming Keyword Spotting (2018)12.10
• Convmixer: Feature Interactive Convolution With Curriculum Learning For Small Footprint And Noisy Far-field Keyword Spotting (2022)12.61
• Efficient Keyword Spotting Using Dilated Convolutions And Gating (2018)13.84
• Predicting Detection Filters For Small Footprint Open-vocabulary Keyword Spotting (2019)9.92
• Small-footprint Keyword Spotting With Graph Convolutional Network (2019)10.48
• Small-footprint Open-vocabulary Keyword Spotting With Quantized LSTM Networks (2020)0.00
• Autokws: Keyword Spotting With Differentiable Architecture Search (2020)9.92
• Streaming Small-footprint Keyword Spotting Using Sequence-to-sequence Models (2017)12.40