DENT-DDSP: Data-efficient Noisy Speech Generator Using Differentiable Digital Signal Processors For Explicit Distortion Modelling And Noise-robust Speech Recognition

Abstract

The performances of automatic speech recognition (ASR) systems degrade drastically under noisy conditions. Explicit distortion modelling (EDM), as a feature compensation step, is able to enhance ASR systems under such conditions by simulating the in-domain noisy speeches from the clean counterparts. Yet, existing distortion models are either non-trainable or unexplainable and often lack controllability and generalization ability. In this paper, we propose a fully explainable and controllable model: DENT-DDSP to achieve EDM. DENT-DDSP utilizes novel differentiable digital signal processing (DDSP) components and requires only 10 seconds of training data to achieve high fidelity. The experiment shows that the simulated noisy data from DENT-DDSP achieves the highest simulation fidelity compared to other baseline models in terms of multi-scale spectral loss (MSSL). Moreover, to validate whether the data simulated by DENT-DDSP are able to replace the scarce in-domain noisy data in the noise-

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyguo2022dent

Related papers

• Beyond Oversmoothing: Evaluating DDPM And MSE For Scalable Speech Synthesis In ASR (2024)0.00
• Bridging The Gap Between Monaural Speech Enhancement And Recognition With Distortion-independent Acoustic Modeling (2019)7.50
• Dynamic Acoustic Compensation And Adaptive Focal Training For Personalized Speech Enhancement (2022)4.52
• DDTSE: Discriminative Diffusion Model For Target Speech Extraction (2023)5.84
• Correlation Distance Skip Connection Denoising Autoencoder (CDSK-DAE) For Speech Feature Enhancement (2019)6.77
• Fast, High-quality And Parameter-efficient Articulatory Synthesis Using Differentiable DSP (2024)2.26
• Rethinking Processing Distortions: Disentangling The Impact Of Speech Enhancement Errors On Speech Recognition Performance (2024)8.35
• Training Data Augmentation For Dysarthric Automatic Speech Recognition By Text-to-dysarthric-speech Synthesis (2024)10.48