Diarizationlm: Speaker Diarization Post-processing With Large Language Models

Abstract

In this paper, we introduce DiarizationLM, a framework to leverage large language models (LLM) to post-process the outputs from a speaker diarization system. Various goals can be achieved with the proposed framework, such as improving the readability of the diarized transcript, or reducing the word diarization error rate (WDER). In this framework, the outputs of the automatic speech recognition (ASR) and speaker diarization systems are represented as a compact textual format, which is included in the prompt to an optionally finetuned LLM. The outputs of the LLM can be used as the refined diarization results with the desired enhancement. As a post-processing step, this framework can be easily applied to any off-the-shelf ASR and speaker diarization systems without retraining existing components. Our experiments show that a finetuned PaLM 2-S model can reduce the WDER by rel. 55.5% on the Fisher telephone conversation dataset, and rel. 44.9% on the Callhome English dataset.

Stats

• citations22
• S2 citations—
• github stars0
• HF likes0
• heat score10.21
• arxiv keywang2024diarizationlm

Related papers

• Llm-based Speaker Diarization Correction: A Generalizable Approach (2024)7.16
• Enhancing Speaker Diarization With Large Language Models: A Contextual Beam Search Approach (2023)7.50
• Lexical Speaker Error Correction: Leveraging Language Models For Speaker Diarization Error Correction (2023)0.00
• Speakerlm: End-to-end Versatile Speaker Diarization And Recognition With Multimodal Large Language Models (2025)5.24
• Latent Class Model With Application To Speaker Diarization (2019)3.58
• SEAL: Speaker Error Correction Using Acoustic-conditioned Large Language Models (2025)0.00
• Improved Large-margin Softmax Loss For Speaker Diarisation (2019)6.34
• Large Language Model Can Transcribe Speech In Multi-talker Scenarios With Versatile Instructions (2024)11.23