Multichannel Loss Function For Supervised Speech Source Separation By Mask-based Beamforming

Abstract

In this paper, we propose two mask-based beamforming methods using a deep neural network (DNN) trained by multichannel loss functions. Beamforming technique using time-frequency (TF)-masks estimated by a DNN have been applied to many applications where TF-masks are used for estimating spatial covariance matrices. To train a DNN for mask-based beamforming, loss functions designed for monaural speech enhancement/separation have been employed. Although such a training criterion is simple, it does not directly correspond to the performance of mask-based beamforming. To overcome this problem, we use multichannel loss functions which evaluate the estimated spatial covariance matrices based on the multichannel Itakura--Saito divergence. DNNs trained by the multichannel loss functions can be applied to construct several beamformers. Experimental results confirmed their effectiveness and robustness to microphone configurations.

Stats

• citations9
• S2 citations—
• github stars0
• HF likes0
• heat score7.50
• arxiv keymasuyama2019multichannel

Related papers

• Sequential Multi-frame Neural Beamforming For Speech Separation And Enhancement (2019)0.00
• Locate And Beamform: Two-dimensional Locating All-neural Beamformer For Multi-channel Speech Separation (2023)3.58
• 3D Neural Beamforming For Multi-channel Speech Separation Against Location Uncertainty (2023)0.00
• Spatial Loss For Unsupervised Multi-channel Source Separation (2022)7.16
• ADL-MVDR: All Deep Learning MVDR Beamformer For Target Speech Separation (2020)15.00
• Multi-channel End-to-end Neural Network For Speech Enhancement, Source Localization, And Voice Activity Detection (2022)0.00
• Embedding And Beamforming: All-neural Causal Beamformer For Multichannel Speech Enhancement (2021)13.05
• Deep Beamforming For Speech Enhancement And Speaker Localization With An Array Response-aware Loss Function (2023)0.00