Improving Semi-supervised End-to-end Automatic Speech Recognition Using Cyclegan And Inter-domain Losses

Abstract

We propose a novel method that combines CycleGAN and inter-domain losses for semi-supervised end-to-end automatic speech recognition. Inter-domain loss targets the extraction of an intermediate shared representation of speech and text inputs using a shared network. CycleGAN uses cycle-consistent loss and the identity mapping loss to preserve relevant characteristics of the input feature after converting from one domain to another. As such, both approaches are suitable to train end-to-end models on unpaired speech-text inputs. In this paper, we exploit the advantages from both inter-domain loss and CycleGAN to achieve better shared representation of unpaired speech and text inputs and thus improve the speech-to-text mapping. Our experimental results on the WSJ eval92 and Voxforge (non English) show 8~8.5% character error rate reduction over the baseline, and the results on LibriSpeech test_clean also show noticeable improvement.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyli2022improving

Related papers

• Improving Noisy Student Training For Low-resource Languages In End-to-end ASR Using Cyclegan And Inter-domain Losses (2024)0.00
• A Multi-discriminator Cyclegan For Unsupervised Non-parallel Speech Domain Adaptation (2018)9.76
• Semi-supervised Sequence-to-sequence ASR Using Unpaired Speech And Text (2019)0.00
• Speech Enhancement Based On Cyclegan With Noise-informed Training (2021)5.84
• Nonparallel High-quality Audio Super Resolution With Domain Adaptation And Resampling Cyclegans (2022)6.77
• Cyclegan-vc2: Improved Cyclegan-based Non-parallel Voice Conversion (2019)17.45
• Cycle-consistency Training For End-to-end Speech Recognition (2018)0.00
• Generative Adversarial Speaker Embedding Networks For Domain Robust End-to-end Speaker Verification (2018)0.00