Turn-to-diarize: Online Speaker Diarization Constrained By Transformer Transducer Speaker Turn Detection

Abstract

In this paper, we present a novel speaker diarization system for streaming on-device applications. In this system, we use a transformer transducer to detect the speaker turns, represent each speaker turn by a speaker embedding, then cluster these embeddings with constraints from the detected speaker turns. Compared with conventional clustering-based diarization systems, our system largely reduces the computational cost of clustering due to the sparsity of speaker turns. Unlike other supervised speaker diarization systems which require annotations of time-stamped speaker labels for training, our system only requires including speaker turn tokens during the transcribing process, which largely reduces the human efforts involved in data collection.

Stats

• citations44
• S2 citations—
• github stars0
• HF likes0
• heat score12.40
• arxiv keyxia2021turn

Related papers

• Discriminative Neural Clustering For Speaker Diarisation (2019)10.07
• Highly Efficient Real-time Streaming And Fully On-device Speaker Diarization With Multi-stage Clustering (2022)0.00
• A Reinforcement Learning Framework For Online Speaker Diarization (2023)0.00
• Transcribe-to-diarize: Neural Speaker Diarization For Unlimited Number Of Speakers Using End-to-end Speaker-attributed ASR (2021)11.49
• Target Speaker Voice Activity Detection With Transformers And Its Integration With End-to-end Neural Diarization (2022)10.48
• Reformulating Speaker Diarization As Community Detection With Emphasis On Topological Structure (2022)5.84
• End-to-end Speaker Diarization As Post-processing (2020)11.08
• Speaker Diarization With LSTM (2017)17.48