End-to-end Diarization For Variable Number Of Speakers With Local-global Networks And Discriminative Speaker Embeddings

Abstract

We present an end-to-end deep network model that performs meeting diarization from single-channel audio recordings. End-to-end diarization models have the advantage of handling speaker overlap and enabling straightforward handling of discriminative training, unlike traditional clustering-based diarization methods. The proposed system is designed to handle meetings with unknown numbers of speakers, using variable-number permutation-invariant cross-entropy based loss functions. We introduce several components that appear to help with diarization performance, including a local convolutional network followed by a global self-attention module, multi-task transfer learning using a speaker identification component, and a sequential approach where the model is refined with a second stage. These are trained and validated on simulated meeting data based on LibriSpeech and LibriTTS datasets; final evaluations are done using LibriCSS, which consists of simulated meetings recorded using real acoust

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymaiti2021end

Related papers

• End-to-end Neural Diarization: Reformulating Speaker Diarization As Simple Multi-label Classification (2020)0.00
• EEND-SS: Joint End-to-end Neural Speaker Diarization And Speech Separation For Flexible Number Of Speakers (2022)10.35
• End-to-end Neural Speaker Diarization With Permutation-free Objectives (2019)21.98
• Towards Neural Diarization For Unlimited Numbers Of Speakers Using Global And Local Attractors (2021)11.29
• End-to-end Speaker Diarization As Post-processing (2020)11.08
• Online Neural Diarization Of Unlimited Numbers Of Speakers Using Global And Local Attractors (2022)10.07
• Joint Training Of Speaker Embedding Extractor, Speech And Overlap Detection For Diarization (2024)2.26
• Integration Of Speech Separation, Diarization, And Recognition For Multi-speaker Meetings: System Description, Comparison, And Analysis (2020)13.23