Multistream CNN For Robust Acoustic Modeling

Abstract

This paper proposes multistream CNN, a novel neural network architecture for robust acoustic modeling in speech recognition tasks. The proposed architecture processes input speech with diverse temporal resolutions by applying different dilation rates to convolutional neural networks across multiple streams to achieve the robustness. The dilation rates are selected from the multiples of a sub-sampling rate of 3 frames. Each stream stacks TDNN-F layers (a variant of 1D CNN), and output embedding vectors from the streams are concatenated then projected to the final layer. We validate the effectiveness of the proposed multistream CNN architecture by showing consistent improvements against Kaldi's best TDNN-F model across various data sets. Multistream CNN improves the WER of the test-other set in the LibriSpeech corpus by 12% (relative). On custom data from ASAPP's production ASR system for a contact center, it records a relative WER improvement of 11% for customer channel audio to prove i

Stats

• citations22
• S2 citations—
• github stars0
• HF likes0
• heat score10.21
• arxiv keyhan2020multistream

Related papers

• State-of-the-art Speech Recognition Using Multi-stream Self-attention With Dilated 1D Convolutions (2019)11.93
• ASAPP-ASR: Multistream CNN And Self-attentive SRU For SOTA Speech Recognition (2020)9.03
• Dilated U-net Based Approach For Multichannel Speech Enhancement From First-order Ambisonics Recordings (2020)0.00
• Decoupled Spatial And Temporal Processing For Resource Efficient Multichannel Speech Enhancement (2024)0.00
• PCNN: A Lightweight Parallel Conformer Neural Network For Efficient Monaural Speech Enhancement (2023)6.77
• Deep Cnns Along The Time Axis With Intermap Pooling For Robustness To Spectral Variations (2016)6.77
• Multichannel Speech Enhancement Without Beamforming (2021)9.41
• Monaural Speech Enhancement Using A Multi-branch Temporal Convolutional Network (2019)3.58