Temporal-spatial Neural Filter: Direction Informed End-to-end Multi-channel Target Speech Separation

Abstract

Target speech separation refers to extracting the target speaker's speech from mixed signals. Despite the recent advances in deep learning based close-talk speech separation, the applications to real-world are still an open issue. Two main challenges are the complex acoustic environment and the real-time processing requirement. To address these challenges, we propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture in reverberant environments, assisted with directional information of the speaker(s). Firstly, against variations brought by complex environment, the key idea is to increase the acoustic representation completeness through the jointly modeling of temporal, spectral and spatial discriminability between the target and interference source. Specifically, temporal, spectral, spatial along with the designed directional features are integrated to create a joint acoustic representation. Secondly, to reduce the latency,

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygu2020temporal

Related papers

• Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters (2023)10.35
• Complex Neural Spatial Filter: Enhancing Multi-channel Target Speech Separation In Complex Domain (2021)11.85
• End-to-end Multi-channel Speech Separation (2019)0.00
• Enhancing End-to-end Multi-channel Speech Separation Via Spatial Feature Learning (2020)12.47
• End-to-end Networks For Supervised Single-channel Speech Separation (2018)0.00
• Spatialnet: Extensively Learning Spatial Information For Multichannel Joint Speech Separation, Denoising And Dereverberation (2023)13.88
• Voicefilter: Targeted Voice Separation By Speaker-conditioned Spectrogram Masking (2018)17.48
• Multi-channel Narrow-band Deep Speech Separation With Full-band Permutation Invariant Training (2021)9.41