Improved Acoustic Word Embeddings For Zero-resource Languages Using Multilingual Transfer

Abstract

Acoustic word embeddings are fixed-dimensional representations of variable-length speech segments. Such embeddings can form the basis for speech search, indexing and discovery systems when conventional speech recognition is not possible. In zero-resource settings where unlabelled speech is the only available resource, we need a method that gives robust embeddings on an arbitrary language. Here we explore multilingual transfer: we train a single supervised embedding model on labelled data from multiple well-resourced languages and then apply it to unseen zero-resource languages. We consider three multilingual recurrent neural network (RNN) models: a classifier trained on the joint vocabularies of all training languages; a Siamese RNN trained to discriminate between same and different words from multiple languages; and a correspondence autoencoder (CAE) RNN trained to reconstruct word pairs. In a word discrimination task on six target languages, all of these models outperform state-of-th

Stats

• citations10
• S2 citations—
• github stars0
• HF likes0
• heat score7.81
• arxiv keykamper2020improved

Related papers

• Multilingual Acoustic Word Embedding Models For Processing Zero-resource Languages (2020)8.09
• Multilingual And Unsupervised Subword Modeling For Zero-resource Languages (2018)7.81
• Discriminative Acoustic Word Embeddings: Recurrent Neural Network-based Approaches (2016)0.00
• Truly Unsupervised Acoustic Word Embeddings Using Weak Top-down Constraints In Encoder-decoder Models (2018)0.00
• Leveraging Multilingual Transfer For Unsupervised Semantic Acoustic Word Embeddings (2023)3.58
• Unsupervised Neural And Bayesian Models For Zero-resource Speech Processing (2017)0.00
• Non-linear Pairwise Language Mappings For Low-resource Multilingual Acoustic Model Fusion (2022)0.00
• Supervised Acoustic Embeddings And Their Transferability Across Languages (2023)0.00