Dualstream Contextual Fusion Network: Efficient Target Speaker Extraction By Leveraging Mixture And Enrollment Interactions

Abstract

Target speaker extraction focuses on extracting a target speech signal from an environment with multiple speakers by leveraging an enrollment. Existing methods predominantly rely on speaker embeddings obtained from the enrollment, potentially disregarding the contextual information and the internal interactions between the mixture and enrollment. In this paper, we propose a novel DualStream Contextual Fusion Network (DCF-Net) in the time-frequency (T-F) domain. Specifically, DualStream Fusion Block (DSFB) is introduced to obtain contextual information and capture the interactions between contextualized enrollment and mixture representation across both spatial and channel dimensions, and then rich and consistent representations are utilized to guide the extraction network for better extraction. Experimental results demonstrate that DCF-Net outperforms state-of-the-art (SOTA) methods, achieving a scale-invariant signal-to-distortion ratio improvement (SI-SDRi) of 21.6 dB on the benchmark

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxue2025dualstream

Related papers

• Target Speaker Extraction By Directly Exploiting Contextual Information In The Time-frequency Domain (2024)9.59
• Speaker-conditioning Single-channel Target Speaker Extraction Using Conformer-based Architectures (2022)6.34
• Multi-channel Target Speech Extraction With Channel Decorrelation And Target Speaker Adaptation (2020)0.00
• End-to-end Target Speaker Speech Recognition Using Context-aware Attention Mechanisms For Challenging Enrollment Scenario (2025)0.00
• Duality Temporal-channel-frequency Attention Enhanced Speaker Representation Learning (2021)5.24
• Improving Channel Decorrelation For Multi-channel Target Speech Extraction (2021)6.34
• X-crossnet: A Complex Spectral Mapping Approach To Target Speaker Extraction With Cross Attention Speaker Embedding Fusion (2024)0.00
• DDTSE: Discriminative Diffusion Model For Target Speech Extraction (2023)5.84