Towards Unsupervised Automatic Speech Recognition Trained By Unaligned Speech And Text Only

Abstract

Automatic speech recognition (ASR) has been widely researched with supervised approaches, while many low-resourced languages lack audio-text aligned data, and supervised methods cannot be applied on them. In this work, we propose a framework to achieve unsupervised ASR on a read English speech dataset, where audio and text are unaligned. In the first stage, each word-level audio segment in the utterances is represented by a vector representation extracted by a sequence-of-sequence autoencoder, in which phonetic information and speaker information are disentangled. Secondly, semantic embeddings of audio segments are trained from the vector representations using a skip-gram model. Last but not the least, an unsupervised method is utilized to transform semantic embeddings of audio segments to text embedding space, and finally the transformed embeddings are mapped to words. With the above framework, we are towards unsupervised ASR trained by unaligned text and speech only.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychen2018towards

Related papers

• Towards Unsupervised Speech Recognition Without Pronunciation Models (2024)0.00
• Almost Unsupervised Text To Speech And Automatic Speech Recognition (2019)0.00
• Unsupervised Automatic Speech Recognition: A Review (2021)13.50
• Almost-unsupervised Speech Recognition With Close-to-zero Resource Based On Phonetic Structures Learned From Very Small Unpaired Speech And Text Data (2018)0.00
• Towards Unsupervised Speech-to-text Translation (2018)0.00
• REBORN: Reinforcement-learned Boundary Segmentation With Iterative Training For Unsupervised ASR (2024)2.26
• Semi-supervised Sequence-to-sequence ASR Using Unpaired Speech And Text (2019)0.00
• Unsupervised Pre-training For Sequence To Sequence Speech Recognition (2019)0.00