Weakly Supervised Spoken Term Discovery Using Cross-lingual Side Information

Abstract

Recent work on unsupervised term discovery (UTD) aims to identify and cluster repeated word-like units from audio alone. These systems are promising for some very low-resource languages where transcribed audio is unavailable, or where no written form of the language exists. However, in some cases it may still be feasible (e.g., through crowdsourcing) to obtain (possibly noisy) text translations of the audio. If so, this information could be used as a source of side information to improve UTD. Here, we present a simple method for rescoring the output of a UTD system using text translations, and test it on a corpus of Spanish audio with English translations. We show that it greatly improves the average precision of the results over a wide range of system configurations and data preprocessing methods.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keybansal2016weakly

Related papers

• Towards Speech-to-text Translation Without Speech Recognition (2017)10.35
• Cross-lingual And Multilingual Spoken Term Detection For Low-resource Indian Languages (2020)0.00
• Spoken Term Detection Methods For Sparse Transcription In Very Low-resource Settings (2021)0.00
• Towards Unsupervised Speech-to-text Translation (2018)0.00
• A Nonparametric Bayesian Approach For Spoken Term Detection By Example Query (2016)0.00
• Leveraging Unsupervised And Weakly-supervised Data To Improve Direct Speech-to-speech Translation (2022)8.35
• Unsupervised Discovery Of Recurring Speech Patterns Using Probabilistic Adaptive Metrics (2020)8.35
• Improving Cascaded Unsupervised Speech Translation With Denoising Back-translation (2023)0.00