Beyond Voice Activity Detection: Hybrid Audio Segmentation For Direct Speech Translation

Abstract

The audio segmentation mismatch between training data and those seen at run-time is a major problem in direct speech translation. Indeed, while systems are usually trained on manually segmented corpora, in real use cases they are often presented with continuous audio requiring automatic (and sub-optimal) segmentation. After comparing existing techniques (VAD-based, fixed-length and hybrid segmentation methods), in this paper we propose enhanced hybrid solutions to produce better results without sacrificing latency. Through experiments on different domains and language pairs, we show that our methods outperform all the other techniques, reducing by at least 30% the gap between the traditional VAD-based approach and optimal manual segmentation.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygaido2021beyond

Related papers

• Speech Segmentation Optimization Using Segmented Bilingual Speech Corpus For End-to-end Speech Translation (2022)5.84
• Don't Discard Fixed-window Audio Segmentation In Speech-to-text Translation (2022)0.00
• Semantic VAD: Low-latency Voice Activity Detection For Speech Interaction (2023)6.34
• Smart Speech Segmentation Using Acousto-linguistic Features With Look-ahead (2022)0.00
• Speech Decomposition Based On A Hybrid Speech Model And Optimal Segmentation (2021)0.00
• Joint Speech And Overlap Detection: A Benchmark Over Multiple Audio Setup And Speech Domains (2023)0.00
• Long-form End-to-end Speech Translation Via Latent Alignment Segmentation (2023)0.00
• Improving Speech Translation Accuracy And Time Efficiency With Fine-tuned Wav2vec 2.0-based Speech Segmentation (2023)3.58