Locate And Beamform: Two-dimensional Locating All-neural Beamformer For Multi-channel Speech Separation

Abstract

Recently, stunning improvements on multi-channel speech separation have been achieved by neural beamformers when direction information is available. However, most of them neglect to utilize speaker's 2-dimensional (2D) location cues contained in mixture signal, which limits the performance when two sources come from close directions. In this paper, we propose an end-to-end beamforming network for 2D location guided speech separation merely given mixture signal. It first estimates discriminable direction and 2D location cues, which imply directions the sources come from in multi views of microphones and their 2D coordinates. These cues are then integrated into location-aware neural beamformer, thus allowing accurate reconstruction of two sources' speech signals. Experiments show that our proposed model not only achieves a comprehensive decent improvement compared to baseline systems, but avoids inferior performance on spatial overlapping cases.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyfu2023locate

Related papers

• 3D Neural Beamforming For Multi-channel Speech Separation Against Location Uncertainty (2023)0.00
• Sequential Multi-frame Neural Beamforming For Speech Separation And Enhancement (2019)0.00
• Towards Unified All-neural Beamforming For Time And Frequency Domain Speech Separation (2022)11.29
• Dual-path Transformer Based Neural Beamformer For Target Speech Extraction (2023)0.00
• Mimo-dbnet: Multi-channel Input And Multiple Outputs Doa-aware Beamforming Network For Speech Separation (2022)0.00
• Embedding And Beamforming: All-neural Causal Beamformer For Multichannel Speech Enhancement (2021)13.05
• Attention-based Neural Beamforming Layers For Multi-channel Speech Recognition (2021)0.00
• Multichannel Loss Function For Supervised Speech Source Separation By Mask-based Beamforming (2019)7.50