Advancing VAD Systems Based On Multi-task Learning With Improved Model Structures

Abstract

In a speech recognition system, voice activity detection (VAD) is a crucial frontend module. Addressing the issues of poor noise robustness in traditional binary VAD systems based on DFSMN, the paper further proposes semantic VAD based on multi-task learning with improved models for real-time and offline systems, to meet specific application requirements. Evaluations on internal datasets show that, compared to the real-time VAD system based on DFSMN, the real-time semantic VAD system based on RWKV achieves relative decreases in CER of 7.0%, DCF of 26.1% and relative improvement in NRR of 19.2%. Similarly, when compared to the offline VAD system based on DFSMN, the offline VAD system based on SAN-M demonstrates relative decreases in CER of 4.4%, DCF of 18.6% and relative improvement in NRR of 3.5%.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzuo2023advancing

Related papers

• Incorporating VAD Into ASR System By Multi-task Learning (2021)4.52
• Semantic VAD: Low-latency Voice Activity Detection For Speech Interaction (2023)6.34
• Speech Enhancement Aided End-to-end Multi-task Learning For Voice Activity Detection (2020)11.49
• Adversarial Multi-task Deep Learning For Noise-robust Voice Activity Detection With Low Algorithmic Delay (2022)2.26
• Vsanet: Real-time Speech Enhancement Based On Voice Activity Detection And Causal Spatial Attention (2023)5.24
• Comparative Analysis Of Personalized Voice Activity Detection Systems: Assessing Real-world Effectiveness (2024)0.00
• MLNET: An Adaptive Multiple Receptive-field Attention Neural Network For Voice Activity Detection (2020)3.58
• Personal VAD: Speaker-conditioned Voice Activity Detection (2019)13.05