The Zero Resource Speech Benchmark 2021: Metrics And Baselines For Unsupervised Spoken Language Modeling

Abstract

We introduce a new unsupervised task, spoken language modeling: the learning of linguistic representations from raw audio signals without any labels, along with the Zero Resource Speech Benchmark 2021: a suite of 4 black-box, zero-shot metrics probing for the quality of the learned models at 4 linguistic levels: phonetics, lexicon, syntax and semantics. We present the results and analyses of a composite baseline made of the concatenation of three unsupervised systems: self-supervised contrastive representation learning (CPC), clustering (k-means) and language modeling (LSTM or BERT). The language models learn on the basis of the pseudo-text derived from clustering the learned representations. This simple pipeline shows better than chance performance on all four metrics, demonstrating the feasibility of spoken language modeling from raw speech. It also yields worse performance compared to text-based 'topline' systems trained on the same data, delineating the space to be explored by more

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynguyen2020the

Related papers

• Self-supervised Language Learning From Raw Audio: Lessons From The Zero Resource Speech Challenge (2022)10.07
• The Zero Resource Speech Challenge 2020: Discovering Discrete Subword And Word Units (2020)11.58
• Multilingual And Unsupervised Subword Modeling For Zero-resource Languages (2018)7.81
• Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languages (2023)7.16
• Unsupervised Neural And Bayesian Models For Zero-resource Speech Processing (2017)0.00
• The Zero Resource Speech Challenge 2019: TTS Without T (2019)13.17
• Information Retrieval For Zerospeech 2021: The Submission By University Of Wroclaw (2021)7.81
• Exploration Of End-to-end Synthesisers Forzero Resource Speech Challenge 2020 (2020)4.52