REBORN: Reinforcement-learned Boundary Segmentation With Iterative Training For Unsupervised ASR

Abstract

Unsupervised automatic speech recognition (ASR) aims to learn the mapping between the speech signal and its corresponding textual transcription without the supervision of paired speech-text data. A word/phoneme in the speech signal is represented by a segment of speech signal with variable length and unknown boundary, and this segmental structure makes learning the mapping between speech and text challenging, especially without paired data. In this paper, we propose REBORN,Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR. REBORN alternates between (1) training a segmentation model that predicts the boundaries of the segmental structures in speech signals and (2) training the phoneme prediction model, whose input is the speech feature segmented by the segmentation model, to predict a phoneme transcription. Since supervised data for training the segmentation model is not available, we use reinforcement learning to train the segmentation model to fa

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keytseng2024reborn

Related papers

• Towards Unsupervised Automatic Speech Recognition Trained By Unaligned Speech And Text Only (2018)0.00
• Unsupervised Automatic Speech Recognition: A Review (2021)13.50
• Towards Unsupervised Speech Recognition Without Pronunciation Models (2024)0.00
• Unsupervised Speech Recognition Via Segmental Empirical Output Distribution Matching (2018)0.00
• Sequence-to-sequence ASR Optimization Via Reinforcement Learning (2017)9.41
• Unpaired Speech Enhancement By Acoustic And Adversarial Supervision For Speech Recognition (2018)10.21
• Catplayinginthesnow: Impact Of Prior Segmentation On A Model Of Visually Grounded Speech (2020)4.52
• Unsupervised Speech Segmentation And Variable Rate Representation Learning Using Segmental Contrastive Predictive Coding (2021)9.92