Advances In Integration Of End-to-end Neural And Clustering-based Diarization For Real Conversational Speech

Abstract

Recently, we proposed a novel speaker diarization method called End-to-End-Neural-Diarization-vector clustering (EEND-vector clustering) that integrates clustering-based and end-to-end neural network-based diarization approaches into one framework. The proposed method combines advantages of both frameworks, i.e. high diarization performance and handling of overlapped speech based on EEND, and robust handling of long recordings with an arbitrary number of speakers based on clustering-based approaches. However, the method was only evaluated so far on simulated 2-speaker meeting-like data. This paper is to (1) report recent advances we made to this framework, including newly introduced robust constrained clustering algorithms, and (2) experimentally show that the method can now significantly outperform competitive diarization methods such as Encoder-Decoder Attractor (EDA)-EEND, on CALLHOME data which comprises real conversational speech data including overlapped speech and an arbitrary n

Stats

• citations48
• S2 citations—
• github stars79
• HF likes0
• heat score16.48
• arxiv keykinoshita2021advances

Related papers

• Integrating End-to-end Neural And Clustering-based Diarization: Getting The Best Of Both Worlds (2020)13.74
• Speakers Unembedded: Embedding-free Approach To Long-form Neural Diarization (2024)3.58
• An Experimental Review Of Speaker Diarization Methods With Application To Two-speaker Conversational Telephone Speech Recordings (2023)8.35
• Tight Integration Of Neural- And Clustering-based Diarization Through Deep Unfolding Of Infinite Gaussian Mixture Model (2022)8.60
• End-to-end Speaker Diarization As Post-processing (2020)11.08
• Improving End-to-end Neural Diarization Using Conversational Summary Representations (2023)0.00
• End-to-end Neural Diarization: Reformulating Speaker Diarization As Simple Multi-label Classification (2020)0.00
• Multi-channel End-to-end Neural Diarization With Distributed Microphones (2021)10.21