Speaker Embedding-aware Neural Diarization: An Efficient Framework For Overlapping Speech Diarization In Meeting Scenarios

Abstract

Overlapping speech diarization has been traditionally treated as a multi-label classification problem. In this paper, we reformulate this task as a single-label prediction problem by encoding multiple binary labels into a single label with the power set, which represents the possible combinations of target speakers. This formulation has two benefits. First, the overlaps of target speakers are explicitly modeled. Second, threshold selection is no longer needed. Through this formulation, we propose the speaker embedding-aware neural diarization (SEND) framework, where a speech encoder, a speaker encoder, two similarity scorers, and a post-processing network are jointly optimized to predict the encoded labels according to the similarities between speech features and speaker embeddings. Experimental results show that SEND has a stable learning process and can be trained on highly overlapped data without extra initialization. More importantly, our method achieves the state-of-the-art perfor

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydu2022speaker

Related papers

• Speaker Embedding-aware Neural Diarization For Flexible Number Of Speakers With Textual Information (2021)0.00
• Leveraging Speaker Embeddings In End-to-end Neural Diarization For Two-speaker Scenarios (2024)0.00
• Overlap-aware Diarization: Resegmentation Using Neural End-to-end Overlapped Speech Detection (2019)13.17
• End-to-end Neural Diarization: Reformulating Speaker Diarization As Simple Multi-label Classification (2020)0.00
• Multi-class Spectral Clustering With Overlaps For Speaker Diarization (2020)10.35
• Compositional Embedding Models For Speaker Identification And Diarization With Simultaneous Speech From 2+ Speakers (2020)3.58
• Geodesic Interpolation Of Frame-wise Speaker Embeddings For The Diarization Of Meeting Scenarios (2024)5.24
• Joint Training Of Speaker Embedding Extractor, Speech And Overlap Detection For Diarization (2024)2.26