Whisper Turns Stronger: Augmenting Wav2vec 2.0 For Superior ASR In Low-resource Languages

Abstract

Approaching Speech-to-Text and Automatic Speech Recognition problems in low-resource languages is notoriously challenging due to the scarcity of validated datasets and the diversity of dialects. Arabic, Russian, and Portuguese exemplify these difficulties, being low-resource languages due to the many dialects of these languages across different continents worldwide. Moreover, the variety of accents and pronunciations of such languages complicate ASR models' success. With the increasing popularity of Deep Learning and Transformers, acoustic models like the renowned Wav2Vec2 have achieved superior performance in the Speech Recognition field compared to state-of-the-art approaches. However, despite Wav2Vec2's improved efficiency over traditional methods, its performance significantly declines for under-represented languages, even though it requires significantly less labeled data. This paper introduces an end-to-end framework that enhances ASR systems fine-tuned on Wav2Vec2 through data a

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyanidjar2024whisper

Related papers

• Whisper-lm: Improving ASR Models With Language Models For Low-resource Languages (2025)3.29
• Frustratingly Easy Data Augmentation For Low-resource ASR (2025)0.00
• M2r-whisper: Multi-stage And Multi-scale Retrieval Augmentation For Enhancing Whisper (2024)6.77
• Voice Conversion Can Improve ASR In Very Low-resource Settings (2021)7.50
• A Noise-robust Self-supervised Pre-training Model Based Speech Representation Learning For Automatic Speech Recognition (2022)11.19
• Weighted Cross-entropy For Low-resource Languages In Multilingual Speech Recognition (2024)6.34
• Whispervc: Decoupled Cross-domain Alignment And Speech Generation For Low-resource Whisper-to-normal Conversion (2025)0.00
• On Scaling Contrastive Representations For Low-resource Speech Recognition (2021)3.58