Spatial Processing Front-end For Distant ASR Exploiting Self-attention Channel Combinator

Abstract

We present a novel multi-channel front-end based on channel shortening with theWeighted Prediction Error (WPE) method followed by a fixed MVDR beamformer used in combination with a recently proposed self-attention-based channel combination (SACC) scheme, for tackling the distant ASR problem. We show that the proposed system used as part of a ContextNet based end-to-end (E2E) ASR system outperforms leading ASR systems as demonstrated by a 21.6% reduction in relative WER on a multi-channel LibriSpeech playback dataset. We also show how dereverberation prior to beamforming is beneficial and compare the WPE method with a modified neural channel shortening approach. An analysis of the non-intrusive estimate of the signal C50 confirms that the 8 channel WPE method provides significant dereverberation of the signals (13.6 dB improvement). We also show how the weights of the SACC system allow the extraction of accurate spatial information which can be beneficial for other speech processing app

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keysharma2022spatial

Related papers

• Self-attention Channel Combinator Frontend For End-to-end Multichannel Far-field Speech Recognition (2021)7.81
• End-to-end Dereverberation, Beamforming, And Speech Recognition With Improved Numerical Stability And Advanced Frontend (2021)10.97
• End-to-end Far-field Speech Recognition With Unified Dereverberation And Beamforming (2020)10.61
• VACE-WPE: Virtual Acoustic Channel Expansion Based On Neural Networks For Weighted Prediction Error-based Speech Dereverberation (2021)3.58
• Task-specific Optimization Of Virtual Channel Linear Prediction-based Speech Dereverberation Front-end For Far-field Speaker Verification (2021)2.26
• Multi-channel Target Speech Extraction With Channel Decorrelation And Target Speaker Adaptation (2020)0.00
• Attention-based Neural Beamforming Layers For Multi-channel Speech Recognition (2021)0.00
• End-to-end Integration Of Speech Recognition, Dereverberation, Beamforming, And Self-supervised Learning Representation (2022)8.60