Learning-based Robust Speaker Counting And Separation With The Aid Of Spatial Coherence

Abstract

A three-stage approach is proposed for speaker counting and speech separation in noisy and reverberant environments. In the spatial feature extraction, a spatial coherence matrix (SCM) is computed using whitened relative transfer functions (wRTFs) across time frames. The global activity functions of each speaker are estimated from a simplex constructed using the eigenvectors of the SCM, while the local coherence functions are computed from the coherence between the wRTFs of a time-frequency bin and the global activity function-weighted RTF of the target speaker. In speaker counting, we use the eigenvalues of the SCM and the maximum similarity of the interframe global activity distributions between two speakers as the input features to the speaker counting network (SCnet). In speaker separation, a global and local activity-driven network (GLADnet) is used to extract each independent speaker signal, which is particularly useful for highly overlapping speech signals. Experimental results

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keyhsu2023learning

Related papers

• Spatialnet: Extensively Learning Spatial Information For Multichannel Joint Speech Separation, Denoising And Dereverberation (2023)13.88
• Crossnet: Leveraging Global, Cross-band, Narrow-band, And Positional Encoding For Single- And Multi-channel Speaker Separation (2024)0.00
• Consep: A Noise- And Reverberation-robust Speech Separation Framework By Magnitude Conditioning (2024)0.00
• Audio-visual Speech Separation And Dereverberation With A Two-stage Multimodal Network (2019)12.47
• Audio-visual Multi-channel Speech Separation, Dereverberation And Recognition (2022)6.77
• Tf-corrnet: Leveraging Spatial Correlation For Continuous Speech Separation (2025)2.26
• Joint Speaker Features Learning For Audio-visual Multichannel Speech Separation And Recognition (2024)0.00
• EEND-SS: Joint End-to-end Neural Speaker Diarization And Speech Separation For Flexible Number Of Speakers (2022)10.35