Mask-weighted Spatial Likelihood Coding For Speaker-independent Joint Localization And Mask Estimation

Abstract

Due to their robustness and flexibility, neural-driven beamformers are a popular choice for speech separation in challenging environments with a varying amount of simultaneous speakers alongside noise and reverberation. Time-frequency masks and relative directions of the speakers regarding a fixed spatial grid can be used to estimate the beamformer's parameters. To some degree, speaker-independence is achieved by ensuring a greater amount of spatial partitions than speech sources. In this work, we analyze how to encode both mask and positioning into such a grid to enable joint estimation of both quantities. We propose mask-weighted spatial likelihood coding and show that it achieves considerable performance in both tasks compared to baseline encodings optimized for either localization or mask estimation. In the same setup, we demonstrate superiority for joint estimation of both quantities. Conclusively, we propose a universal approach which can replace an upstream sound source localiza

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykienegger2024mask

Related papers

• Locate And Beamform: Two-dimensional Locating All-neural Beamformer For Multi-channel Speech Separation (2023)3.58
• Speaker Adapted Beamforming For Multi-channel Automatic Speech Recognition (2018)5.84
• 3D Neural Beamforming For Multi-channel Speech Separation Against Location Uncertainty (2023)0.00
• Statistical Beamformer Exploiting Non-stationarity And Sparsity With Spatially Constrained ICA For Robust Speech Recognition (2023)0.00
• Multichannel Loss Function For Supervised Speech Source Separation By Mask-based Beamforming (2019)7.50
• Sequential Multi-frame Neural Beamforming For Speech Separation And Enhancement (2019)0.00
• Multiple-speaker Localization Based On Direct-path Features And Likelihood Maximization With Spatial Sparsity Regularization (2016)11.85
• Optimization Of Speaker Extraction Neural Network With Magnitude And Temporal Spectrum Approximation Loss (2019)11.29