An Approach To Optimize Inference Of The DIART Speaker Diarization Pipeline

Abstract

Speaker diarization answers the question "who spoke when" for an audio file. In some diarization scenarios, low latency is required for transcription. Speaker diarization with low latency is referred to as online speaker diarization. The DIART pipeline is an online speaker diarization system. It consists of a segmentation and an embedding model. The embedding model has the largest share of the overall latency. The aim of this paper is to optimize the inference latency of the DIART pipeline. Different inference optimization methods such as knowledge distilation, pruning, quantization and layer fusion are applied to the embedding model of the pipeline. It turns out that knowledge distillation optimizes the latency, but has a negative effect on the accuracy. Quantization and layer fusion also have a positive influence on the latency without worsening the accuracy. Pruning, on the other hand, does not improve latency.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyaperdannier2024an

Related papers

• Systematic Evaluation Of Online Speaker Diarization Systems Regarding Their Latency (2024)0.00
• Overlap-aware Low-latency Online Speaker Diarization Based On End-to-end Local Segmentation (2021)10.35
• A Reinforcement Learning Framework For Online Speaker Diarization (2023)0.00
• Low-latency Online Speaker Diarization With Graph-based Label Generation (2021)8.60
• Designing An Effective Metric Learning Pipeline For Speaker Diarization (2018)8.60
• Enhancements For Audio-only Diarization Systems (2019)0.00
• Joint Training Of Speaker Embedding Extractor, Speech And Overlap Detection For Diarization (2024)2.26
• Exploring Speaker-related Information In Spoken Language Understanding For Better Speaker Diarization (2023)0.00