Small Footprint Multi-channel Convmixer For Keyword Spotting With Centroid Based Awareness

Abstract

It is critical for a keyword spotting model to have a small footprint as it typically runs on-device with low computational resources. However, maintaining the previous SOTA performance with reduced model size is challenging. In addition, a far-field and noisy environment with multiple signals interference aggravates the problem causing the accuracy to degrade significantly. In this paper, we present a multi-channel ConvMixer for speech command recognitions. The novel architecture introduces an additional audio channel mixing for channel audio interaction in a multi-channel audio setting to achieve better noise-robust features with more efficient computation. Besides, we proposed a centroid based awareness component to enhance the system by equipping it with additional spatial geometry information in the latent feature projection space. We evaluate our model using the new MISP challenge 2021 dataset. Our model achieves significant improvement against the official baseline with a 55% ga

Stats

• citations13
• S2 citations—
• github stars0
• HF likes0
• heat score8.60
• arxiv keyng2022small

Related papers

• Convmixer: Feature Interactive Convolution With Curriculum Learning For Small Footprint And Noisy Far-field Keyword Spotting (2022)12.61
• DCCRN-KWS: An Audio Bias Based Model For Noise Robust Small-footprint Keyword Spotting (2023)5.24
• A Monaural Speech Enhancement Method For Robust Small-footprint Keyword Spotting (2019)0.00
• Challenges And Insights: Exploring 3D Spatial Features And Complex Networks On The MISP Dataset (2023)0.00
• Unsupervised Multi-channel Separation And Adaptation (2023)4.52
• Small-footprint Keyword Spotting With Multi-scale Temporal Convolution (2020)0.00
• Multi-task Network For Noise-robust Keyword Spotting And Speaker Verification Using Ctc-based Soft VAD And Global Query Attention (2020)9.41
• Automatic Channel Selection And Spatial Feature Integration For Multi-channel Speech Recognition Across Various Array Topologies (2023)8.09