CMGAN: Conformer-based Metric GAN For Speech Enhancement

Abstract

Recently, convolution-augmented transformer (Conformer) has achieved promising performance in automatic speech recognition (ASR) and time-domain speech enhancement (SE), as it can capture both local and global dependencies in the speech signal. In this paper, we propose a conformer-based metric generative adversarial network (CMGAN) for SE in the time-frequency (TF) domain. In the generator, we utilize two-stage conformer blocks to aggregate all magnitude and complex spectrogram information by modeling both time and frequency dependencies. The estimation of magnitude and complex spectrogram is decoupled in the decoder stage and then jointly incorporated to reconstruct the enhanced speech. In addition, a metric discriminator is employed to further improve the quality of the enhanced estimated speech by optimizing the generator with respect to a corresponding evaluation score. Quantitative analysis on Voice Bank+DEMAND dataset indicates the capability of CMGAN in outperforming various pr

Stats

• citations103
• S2 citations—
• github stars0
• HF likes0
• heat score15.13
• arxiv keycao2022cmgan

Related papers

• CMGAN: Conformer-based Metric-gan For Monaural Speech Enhancement (2022)14.80
• Conditional Generative Adversarial Networks For Speech Enhancement And Noise-robust Speaker Verification (2017)16.03
• DCCRGAN: Deep Complex Convolution Recurrent Generator Adversarial Network For Speech Enhancement (2020)0.00
• SEGAN: Speech Enhancement Generative Adversarial Network (2017)21.85
• Tdcgan: Temporal Dilated Convolutional Generative Adversarial Network For End-to-end Speech Enhancement (2020)0.00
• Metricgan: Generative Adversarial Networks Based Black-box Metric Scores Optimization For Speech Enhancement (2019)0.00
• Multi-metric Optimization Using Generative Adversarial Networks For Near-end Speech Intelligibility Enhancement (2021)8.60
• Towards Generalized Speech Enhancement With Generative Adversarial Networks (2019)10.35