Exploring Speech Recognition, Translation, And Understanding With Discrete Speech Units: A Comparative Study
2023 · Xuankai Chang, Brian Yan, Kwanghee Choi, et al.
Abstract
Speech signals, typically sampled at rates in the tens of thousands per second, contain redundancies, evoking inefficiencies in sequence modeling. High-dimensional speech features such as spectrograms are often used as the input for the subsequent model. However, they can still be redundant. Recent investigations proposed the use of discrete speech units derived from self-supervised learning representations, which significantly compresses the size of speech data. Applying various methods, such as de-duplication and subword modeling, can further compress the speech sequence length. Hence, training time is significantly reduced while retaining notable performance. In this study, we undertake a comprehensive and systematic exploration into the application of discrete units within end-to-end speech processing models. Experiments on 12 automatic speech recognition, 3 speech translation, and 1 spoken language understanding corpora demonstrate that discrete units achieve reasonably good resul
Authors
(none)
Tags
Stats
Related papers
- Analysing Discrete Self Supervised Speech Representation For Spoken Language Modeling (2023)12.86
- Analyzing Speech Unit Selection For Textless Speech-to-speech Translation (2024)0.00
- Discreteslu: A Large Language Model With Self-supervised Discrete Speech Units For Spoken Language Understanding (2024)5.84
- A Comparison Of Discrete And Soft Speech Units For Improved Voice Conversion (2021)20.25
- Combining Adversarial Training And Disentangled Speech Representation For Robust Zero-resource Subword Modeling (2019)7.16
- Exploration Of Efficient End-to-end ASR Using Discretized Input From Self-supervised Learning (2023)12.02
- Efficient Extraction Of Noise-robust Discrete Units From Self-supervised Speech Models (2024)0.00
- Unity: Two-pass Direct Speech-to-speech Translation With Discrete Units (2022)9.59