Self-supervised Language Learning From Raw Audio: Lessons From The Zero Resource Speech Challenge

Abstract

Recent progress in self-supervised or unsupervised machine learning has opened the possibility of building a full speech processing system from raw audio without using any textual representations or expert labels such as phonemes, dictionaries or parse trees. The contribution of the Zero Resource Speech Challenge series since 2015 has been to break down this long-term objective into four well-defined tasks -- Acoustic Unit Discovery, Spoken Term Discovery, Discrete Resynthesis, and Spoken Language Modeling -- and introduce associated metrics and benchmarks enabling model comparison and cumulative progress. We present an overview of the six editions of this challenge series since 2015, discuss the lessons learned, and outline the areas which need more work or give puzzling results.

Stats

• citations21
• S2 citations—
• github stars0
• HF likes0
• heat score10.07
• arxiv keydunbar2022self

Related papers

• The Zero Resource Speech Challenge 2020: Discovering Discrete Subword And Word Units (2020)11.58
• The Zero Resource Speech Benchmark 2021: Metrics And Baselines For Unsupervised Spoken Language Modeling (2020)0.00
• The Zero Resource Speech Challenge 2019: TTS Without T (2019)13.17
• Exploration Of End-to-end Synthesisers Forzero Resource Speech Challenge 2020 (2020)4.52
• Unsupervised Neural And Bayesian Models For Zero-resource Speech Processing (2017)0.00
• Information Retrieval For Zerospeech 2021: The Submission By University Of Wroclaw (2021)7.81
• Multilingual And Unsupervised Subword Modeling For Zero-resource Languages (2018)7.81
• An Iterative Deep Learning Framework For Unsupervised Discovery Of Speech Features And Linguistic Units With Applications On Spoken Term Detection (2016)0.00