Speech Enhancement Aided End-to-end Multi-task Learning For Voice Activity Detection
2020 · Xu Tan, Xiao-Lei Zhang
Abstract
Robust voice activity detection (VAD) is a challenging task in low signal-to-noise (SNR) environments. Recent studies show that speech enhancement is helpful to VAD, but the performance improvement is limited. To address this issue, here we propose a speech enhancement aided end-to-end multi-task model for VAD. The model has two decoders, one for speech enhancement and the other for VAD. The two decoders share the same encoder and speech separation network. Unlike the direct thought that takes two separated objectives for VAD and speech enhancement respectively, here we propose a new joint optimization objective -- VAD-masked scale-invariant source-to-distortion ratio (mSI-SDR). mSI-SDR uses VAD information to mask the output of the speech enhancement decoder in the training process. It makes the VAD and speech enhancement tasks jointly optimized not only at the shared encoder and separation network, but also at the objective level. It also satisfies real-time working requirement theor
Authors
(none)
Tags
Stats
Related papers
- Incorporating VAD Into ASR System By Multi-task Learning (2021)4.52
- Adversarial Multi-task Deep Learning For Noise-robust Voice Activity Detection With Low Algorithmic Delay (2022)2.26
- Advancing VAD Systems Based On Multi-task Learning With Improved Model Structures (2023)0.00
- Vsanet: Real-time Speech Enhancement Based On Voice Activity Detection And Causal Spatial Attention (2023)5.24
- Channel-combination Algorithms For Robust Distant Voice Activity And Overlapped Speech Detection (2024)6.34
- Voice Activity Detection: Merging Source And Filter-based Information (2019)13.50
- Multi-input Multi-output Target-speaker Voice Activity Detection For Unified, Flexible, And Robust Audio-visual Speaker Diarization (2024)0.00
- Joint Speech And Overlap Detection: A Benchmark Over Multiple Audio Setup And Speech Domains (2023)0.00