Noise-robust Target-speaker Voice Activity Detection Through Self-supervised Pretraining

Abstract

Target-Speaker Voice Activity Detection (TS-VAD) is the task of detecting the presence of speech from a known target-speaker in an audio frame. Recently, deep neural network-based models have shown good performance in this task. However, training these models requires extensive labelled data, which is costly and time-consuming to obtain, particularly if generalization to unseen environments is crucial. To mitigate this, we propose a causal, Self-Supervised Learning (SSL) pretraining framework, called Denoising Autoregressive Predictive Coding (DN-APC), to enhance TS-VAD performance in noisy conditions. We also explore various speaker conditioning methods and evaluate their performance under different noisy conditions. Our experiments show that DN-APC improves performance in noisy conditions, with a general improvement of approx. 2% in both seen and unseen noise. Additionally, we find that FiLM conditioning provides the best overall performance. Representation analysis via tSNE plots re

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybovbjerg2025noise

Related papers

• Self-supervised Pretraining For Robust Personalized Voice Activity Detection In Adverse Conditions (2023)6.34
• Adversarial Multi-task Deep Learning For Noise-robust Voice Activity Detection With Low Algorithmic Delay (2022)2.26
• Cross-domain Voice Activity Detection With Self-supervised Representations (2022)0.00
• Personal VAD: Speaker-conditioned Voice Activity Detection (2019)13.05
• Self-adaptive Soft Voice Activity Detection Using Deep Neural Networks For Robust Speaker Verification (2019)6.77
• Target-speaker Voice Activity Detection Via Sequence-to-sequence Prediction (2022)11.19
• Target-speaker Voice Activity Detection With Improved I-vector Estimation For Unknown Number Of Speaker (2021)10.97
• Profile-error-tolerant Target-speaker Voice Activity Detection (2023)6.77