Towards Learning A Universal Non-semantic Representation Of Speech

Abstract

The ultimate goal of transfer learning is to reduce labeled data requirements by exploiting a pre-existing embedding model trained for different datasets or tasks. The visual and language communities have established benchmarks to compare embeddings, but the speech community has yet to do so. This paper proposes a benchmark for comparing speech representations on non-semantic tasks, and proposes a representation based on an unsupervised triplet-loss objective. The proposed representation outperforms other representations on the benchmark, and even exceeds state-of-the-art performance on a number of transfer learning tasks. The embedding is trained on a publicly available dataset, and it is tested on a variety of low-resource downstream tasks, including personalization tasks and medical domain. The benchmark, models, and evaluation code are publicly released.

Stats

• citations83
• S2 citations—
• github stars0
• HF likes0
• heat score14.43
• arxiv keyshor2020towards

Related papers

• Unispeech: Unified Speech Representation Learning With Labeled And Unlabeled Data (2021)0.00
• Word-level Embeddings For Cross-task Transfer Learning In Speech Processing (2019)5.24
• SUPERB-SG: Enhanced Speech Processing Universal Performance Benchmark For Semantic And Generative Capabilities (2022)13.34
• Learning Efficient Representations For Keyword Spotting With Triplet Loss (2021)11.76
• Learning Problem-agnostic Speech Representations From Multiple Self-supervised Tasks (2019)15.54
• Unispeech-sat: Universal Speech Representation Learning With Speaker Aware Pre-training (2021)0.00
• An Unsupervised Autoregressive Model For Speech Representation Learning (2019)17.26
• Unsupervised Transfer Learning For Spoken Language Understanding In Intelligent Agents (2018)0.00