A Universally-deployable ASR Frontend For Joint Acoustic Echo Cancellation, Speech Enhancement, And Voice Separation

Abstract

Recent work has shown that it is possible to train a single model to perform joint acoustic echo cancellation (AEC), speech enhancement, and voice separation, thereby serving as a unified frontend for robust automatic speech recognition (ASR). The joint model uses contextual information, such as a reference of the playback audio, noise context, and speaker embedding. In this work, we propose a number of novel improvements to such a model. First, we improve the architecture of the Cross-Attention Conformer that is used to ingest noise context into the model. Second, we generalize the model to be able to handle varying lengths of noise context. Third, we propose Signal Dropout, a novel strategy that models missing contextual information. In the absence of one or more signals, the proposed model performs nearly as well as task-specific models trained without these signals; and when such signals are present, our system compares well against systems that require all context signals. Over th

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyomalley2022a

Related papers

• A Conformer-based ASR Frontend For Joint Acoustic Echo Cancellation, Speech Enhancement And Speech Separation (2021)9.23
• Neuralecho: A Self-attentive Recurrent Neural Network For Unified Acoustic Echo Suppression And Speech Enhancement (2022)0.00
• Real-time Joint Personalized Speech Enhancement And Acoustic Echo Cancellation (2022)4.52
• Towards Decoupling Frontend Enhancement And Backend Recognition In Monaural Robust ASR (2024)4.52
• Joint Neural AEC And Beamforming With Double-talk Detection (2021)3.58
• Deepvqe: Real Time Deep Voice Quality Enhancement For Joint Acoustic Echo Cancellation, Noise Suppression And Dereverberation (2023)0.00
• 4D ASR: Joint Modeling Of CTC, Attention, Transducer, And Mask-predict Decoders (2022)7.50
• Joint Training Of Speech Enhancement And Self-supervised Model For Noise-robust ASR (2022)0.00