Incorporating VAD Into ASR System By Multi-task Learning

Abstract

When we use End-to-end automatic speech recognition (E2E-ASR) system for real-world applications, a voice activity detection (VAD) system is usually needed to improve the performance and to reduce the computational cost by discarding non-speech parts in the audio. Usually ASR and VAD systems are trained and utilized independently to each other. In this paper, we present a novel multi-task learning (MTL) framework that incorporates VAD into the ASR system. The proposed system learns ASR and VAD jointly in the training stage. With the assistance of VAD, the ASR performance improves as its connectionist temporal classification (CTC) loss function can leverage the VAD alignment information. In the inference stage, the proposed system removes non-speech parts at low computational cost and recognizes speech parts with high robustness. Experimental results on segmented speech data show that by utilizing VAD information, the proposed method outperforms the baseline ASR system on both English a

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyli2021incorporating

Related papers

• Speech Enhancement Aided End-to-end Multi-task Learning For Voice Activity Detection (2020)11.49
• Advancing VAD Systems Based On Multi-task Learning With Improved Model Structures (2023)0.00
• End-to-end Automatic Speech Recognition Integrated With Ctc-based Voice Activity Detection (2020)11.76
• Adversarial Multi-task Deep Learning For Noise-robust Voice Activity Detection With Low Algorithmic Delay (2022)2.26
• E2e-based Multi-task Learning Approach To Joint Speech And Accent Recognition (2021)0.00
• Semantic VAD: Low-latency Voice Activity Detection For Speech Interaction (2023)6.34
• Multi-task Learning Of Deep Neural Networks For Audio Visual Automatic Speech Recognition (2017)0.00
• Tandem Multitask Training Of Speaker Diarisation And Speech Recognition For Meeting Transcription (2022)7.81