Speech Segmentation Optimization Using Segmented Bilingual Speech Corpus For End-to-end Speech Translation

Abstract

Speech segmentation, which splits long speech into short segments, is essential for speech translation (ST). Popular VAD tools like WebRTC VAD have generally relied on pause-based segmentation. Unfortunately, pauses in speech do not necessarily match sentence boundaries, and sentences can be connected by a very short pause that is difficult to detect by VAD. In this study, we propose a speech segmentation method using a binary classification model trained using a segmented bilingual speech corpus. We also propose a hybrid method that combines VAD and the above speech segmentation method. Experimental results revealed that the proposed method is more suitable for cascade and end-to-end ST systems than conventional segmentation methods. The hybrid approach further improved the translation performance.

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyfukuda2022speech

Related papers

• Beyond Voice Activity Detection: Hybrid Audio Segmentation For Direct Speech Translation (2021)0.00
• Improving Speech Translation Accuracy And Time Efficiency With Fine-tuned Wav2vec 2.0-based Speech Segmentation (2023)3.58
• Don't Discard Fixed-window Audio Segmentation In Speech-to-text Translation (2022)0.00
• Long-form End-to-end Speech Translation Via Latent Alignment Segmentation (2023)0.00
• Smart Speech Segmentation Using Acousto-linguistic Features With Look-ahead (2022)0.00
• Impact Of Encoding And Segmentation Strategies On End-to-end Simultaneous Speech Translation (2021)4.52
• Segaugment: Maximizing The Utility Of Speech Translation Data With Segmentation-based Augmentations (2022)0.00
• End-to-end Simultaneous Speech Translation With Differentiable Segmentation (2023)7.16