An Unsupervised Probability Model For Speech-to-translation Alignment Of Low-resource Languages

Abstract

For many low-resource languages, spoken language resources are more likely to be annotated with translations than with transcriptions. Translated speech data is potentially valuable for documenting endangered languages or for training speech translation systems. A first step towards making use of such data would be to automatically align spoken words with their translations. We present a model that combines Dyer et al.'s reparameterization of IBM Model 2 (fast-align) and k-means clustering using Dynamic Time Warping as a distance metric. The two components are trained jointly using expectation-maximization. In an extremely low-resource scenario, our model performs significantly better than both a neural model and a strong baseline.

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keyanastasopoulos2016an

Related papers

• Leveraging Translations For Speech Transcription In Low-resource Settings (2018)6.77
• Multilingual And Unsupervised Subword Modeling For Zero-resource Languages (2018)7.81
• Towards Speech-to-text Translation Without Speech Recognition (2017)10.35
• Non-linear Pairwise Language Mappings For Low-resource Multilingual Acoustic Model Fusion (2022)0.00
• Exploiting Cross-lingual Speaker And Phonetic Diversity For Unsupervised Subword Modeling (2019)6.77
• Efficient Neural Speech Synthesis For Low-resource Languages Through Multilingual Modeling (2020)8.60
• Learning Cross-lingual Mappings For Data Augmentation To Improve Low-resource Speech Recognition (2023)0.00
• Sample, Translate, Recombine: Leveraging Audio Alignments For Data Augmentation In End-to-end Speech Translation (2022)7.81