Compositional Embedding Models For Speaker Identification And Diarization With Simultaneous Speech From 2+ Speakers

Abstract

We propose a new method for speaker diarization that can handle overlapping speech with 2+ people. Our method is based on compositional embeddings [1]: Like standard speaker embedding methods such as x-vector [2], compositional embedding models contain a function f that separates speech from different speakers. In addition, they include a composition function g to compute set-union operations in the embedding space so as to infer the set of speakers within the input audio. In an experiment on multi-person speaker identification using synthesized LibriSpeech data, the proposed method outperforms traditional embedding methods that are only trained to separate single speakers (not speaker sets). In a speaker diarization experiment on the AMI Headset Mix corpus, we achieve state-of-the-art accuracy (DER=22.93%), slightly higher than the previous best result (23.82% from [3]).

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyli2020compositional

Related papers

• Speaker Embedding-aware Neural Diarization: An Efficient Framework For Overlapping Speech Diarization In Meeting Scenarios (2022)0.00
• Speaker Diarisation Using 2D Self-attentive Combination Of Embeddings (2019)9.92
• Multi-class Spectral Clustering With Overlaps For Speaker Diarization (2020)10.35
• Geodesic Interpolation Of Frame-wise Speaker Embeddings For The Diarization Of Meeting Scenarios (2024)5.24
• Leveraging Speaker Embeddings In End-to-end Neural Diarization For Two-speaker Scenarios (2024)0.00
• Speaker Embedding-aware Neural Diarization For Flexible Number Of Speakers With Textual Information (2021)0.00
• Integrating Audio, Visual, And Semantic Information For Enhanced Multimodal Speaker Diarization (2024)0.00
• Unified Modeling Of Multi-talker Overlapped Speech Recognition And Diarization With A Sidecar Separator (2023)7.50