Complex Spectrogram Enhancement By Convolutional Neural Network With Multi-metrics Learning

Abstract

This paper aims to address two issues existing in the current speech enhancement methods: 1) the difficulty of phase estimations; 2) a single objective function cannot consider multiple metrics simultaneously. To solve the first problem, we propose a novel convolutional neural network (CNN) model for complex spectrogram enhancement, namely estimating clean real and imaginary (RI) spectrograms from noisy ones. The reconstructed RI spectrograms are directly used to synthesize enhanced speech waveforms. In addition, since log-power spectrogram (LPS) can be represented as a function of RI spectrograms, its reconstruction is also considered as another target. Thus a unified objective function, which combines these two targets (reconstruction of RI spectrograms and LPS), is equivalent to simultaneously optimizing two commonly used objective metrics: segmental signal-to-noise ratio (SSNR) and logspectral distortion (LSD). Therefore, the learning process is called multi-metrics learning (MML).

Stats

• citations118
• S2 citations—
• github stars0
• HF likes0
• heat score15.57
• arxiv keyfu2017complex

Related papers

• Magnitude-and-phase-aware Speech Enhancement With Parallel Sequence Modeling (2023)3.58
• Complex Spectral Mapping With Attention Based Convolution Recurrent Neural Network For Speech Enhancement (2021)0.00
• Multi-cmgan+/+: Leveraging Multi-objective Speech Quality Metric Prediction For Speech Enhancement (2023)0.00
• Mp-senet: A Speech Enhancement Model With Parallel Denoising Of Magnitude And Phase Spectra (2023)15.51
• DCCRN: Deep Complex Convolution Recurrent Network For Phase-aware Speech Enhancement (2020)20.78
• Primek-net: Multi-scale Spectral Learning Via Group Prime-kernel Convolutional Neural Networks For Single Channel Speech Enhancement (2025)0.00
• Leveraging Joint Spectral And Spatial Learning With MAMBA For Multichannel Speech Enhancement (2024)0.00
• Monaural Speech Enhancement Using A Multi-branch Temporal Convolutional Network (2019)3.58