Joint Speech And Overlap Detection: A Benchmark Over Multiple Audio Setup And Speech Domains

Abstract

Voice activity and overlapped speech detection (respectively VAD and OSD) are key pre-processing tasks for speaker diarization. The final segmentation performance highly relies on the robustness of these sub-tasks. Recent studies have shown VAD and OSD can be trained jointly using a multi-class classification model. However, these works are often restricted to a specific speech domain, lacking information about the generalization capacities of the systems. This paper proposes a complete and new benchmark of different VAD and OSD models, on multiple audio setups (single/multi-channel) and speech domains (e.g. media, meeting...). Our 2/3-class systems, which combine a Temporal Convolutional Network with speech representations adapted to the setup, outperform state-of-the-art results. We show that the joint training of these two tasks offers similar performances in terms of F1-score to two dedicated VAD and OSD systems while reducing the training cost. This unique architecture can also be

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylebourdais2023joint

Related papers

• Channel-combination Algorithms For Robust Distant Voice Activity And Overlapped Speech Detection (2024)6.34
• Joint Training Of Speaker Embedding Extractor, Speech And Overlap Detection For Diarization (2024)2.26
• Large-scale Learning On Overlapped Speech Detection: New Benchmark And New General System (2023)0.00
• Speech Enhancement Aided End-to-end Multi-task Learning For Voice Activity Detection (2020)11.49
• Multi-input Multi-output Target-speaker Voice Activity Detection For Unified, Flexible, And Robust Audio-visual Speaker Diarization (2024)0.00
• Joint Domain Adaptation And Speech Bandwidth Extension Using Time-domain Gans For Speaker Verification (2022)4.52
• Joint Training Or Not: An Exploration Of Pre-trained Speech Models In Audio-visual Speaker Diarization (2023)0.00
• Cross-channel Attention-based Target Speaker Voice Activity Detection: Experimental Results For M2met Challenge (2022)10.07