Multilingual Acoustic Word Embedding Models For Processing Zero-resource Languages

Abstract

Acoustic word embeddings are fixed-dimensional representations of variable-length speech segments. In settings where unlabelled speech is the only available resource, such embeddings can be used in "zero-resource" speech search, indexing and discovery systems. Here we propose to train a single supervised embedding model on labelled data from multiple well-resourced languages and then apply it to unseen zero-resource languages. For this transfer learning approach, we consider two multilingual recurrent neural network models: a discriminative classifier trained on the joint vocabularies of all training languages, and a correspondence autoencoder trained to reconstruct word pairs. We test these using a word discrimination task on six target zero-resource languages. When trained on seven well-resourced languages, both models perform similarly and outperform unsupervised models trained on the zero-resource languages. With just a single training language, the second model works better, but p

Stats

• citations11
• S2 citations—
• github stars0
• HF likes0
• heat score8.09
• arxiv keykamper2020multilingual

Related papers

• Improved Acoustic Word Embeddings For Zero-resource Languages Using Multilingual Transfer (2020)7.81
• Multilingual And Unsupervised Subword Modeling For Zero-resource Languages (2018)7.81
• Truly Unsupervised Acoustic Word Embeddings Using Weak Top-down Constraints In Encoder-decoder Models (2018)0.00
• Unsupervised Neural And Bayesian Models For Zero-resource Speech Processing (2017)0.00
• Discriminative Acoustic Word Embeddings: Recurrent Neural Network-based Approaches (2016)0.00
• Non-linear Pairwise Language Mappings For Low-resource Multilingual Acoustic Model Fusion (2022)0.00
• Leveraging Multilingual Transfer For Unsupervised Semantic Acoustic Word Embeddings (2023)3.58
• Exploiting Cross-lingual Speaker And Phonetic Diversity For Unsupervised Subword Modeling (2019)6.77