Semantic VAD: Low-latency Voice Activity Detection For Speech Interaction

Abstract

For speech interaction, voice activity detection (VAD) is often used as a front-end. However, traditional VAD algorithms usually need to wait for a continuous tail silence to reach a preset maximum duration before segmentation, resulting in a large latency that affects user experience. In this paper, we propose a novel semantic VAD for low-latency segmentation. Different from existing methods, a frame-level punctuation prediction task is added to the semantic VAD, and the artificial endpoint is included in the classification category in addition to the often-used speech presence and absence. To enhance the semantic information of the model, we also incorporate an automatic speech recognition (ASR) related semantic loss. Evaluations on an internal dataset show that the proposed method can reduce the average latency by 53.3% without significant deterioration of character error rate in the back-end ASR compared to the traditional VAD approach.

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keyshi2023semantic

Related papers

• Advancing VAD Systems Based On Multi-task Learning With Improved Model Structures (2023)0.00
• Adversarial Multi-task Deep Learning For Noise-robust Voice Activity Detection With Low Algorithmic Delay (2022)2.26
• Personal VAD: Speaker-conditioned Voice Activity Detection (2019)13.05
• Incorporating VAD Into ASR System By Multi-task Learning (2021)4.52
• End-to-end Automatic Speech Recognition Integrated With Ctc-based Voice Activity Detection (2020)11.76
• Beyond Voice Activity Detection: Hybrid Audio Segmentation For Direct Speech Translation (2021)0.00
• Speech Enhancement Aided End-to-end Multi-task Learning For Voice Activity Detection (2020)11.49
• Comparative Analysis Of Personalized Voice Activity Detection Systems: Assessing Real-world Effectiveness (2024)0.00