Speaker Diarisation Using 2D Self-attentive Combination Of Embeddings

Abstract

Speaker diarisation systems often cluster audio segments using speaker embeddings such as i-vectors and d-vectors. Since different types of embeddings are often complementary, this paper proposes a generic framework to improve performance by combining them into a single embedding, referred to as a c-vector. This combination uses a 2-dimensional (2D) self-attentive structure, which extends the standard self-attentive layer by averaging not only across time but also across different types of embeddings. Two types of 2D self-attentive structure in this paper are the simultaneous combination and the consecutive combination, adopting a single and multiple self-attentive layers respectively. The penalty term in the original self-attentive layer which is jointly minimised with the objective function to encourage diversity of annotation vectors is also modified to obtain not only different local peaks but also the overall trends in the multiple annotation vectors. Experiments on the AMI meetin

Stats

• citations20
• S2 citations—
• github stars0
• HF likes0
• heat score9.92
• arxiv keysun2019speaker

Related papers

• Combination Of Deep Speaker Embeddings For Diarisation (2020)8.60
• Multi-scale Speaker Embedding-based Graph Attention Networks For Speaker Diarisation (2021)8.35
• Compositional Embedding Models For Speaker Identification And Diarization With Simultaneous Speech From 2+ Speakers (2020)3.58
• Speaker Diarization With LSTM (2017)17.48
• Spectral Clustering-aware Learning Of Embeddings For Speaker Diarisation (2022)2.26
• Speaker Embeddings With Weakly Supervised Voice Activity Detection For Efficient Speaker Diarization (2024)2.26
• Integrating Audio, Visual, And Semantic Information For Enhanced Multimodal Speaker Diarization (2024)0.00
• Advancing The Dimensionality Reduction Of Speaker Embeddings For Speaker Diarisation: Disentangling Noise And Informing Speech Activity (2021)2.26