Contextual Biasing To Improve Domain-specific Custom Vocabulary Audio Transcription Without Explicit Fine-tuning Of Whisper Model
2024 Β· Vishakha Lall, Yisi Liu
Abstract
OpenAI's Whisper Automated Speech Recognition model excels in generalizing across diverse datasets and domains. However, this broad adaptability can lead to diminished performance in tasks requiring recognition of specific vocabularies. Addressing this challenge typically involves fine-tuning the model, which demands extensive labeled audio data that is often difficult to acquire and unavailable for specific domains. In this study, we propose a method to enhance transcription accuracy without explicit fine-tuning or altering model parameters, using a relatively small training dataset. Our method leverages contextual biasing, to direct Whisper model's output towards a specific vocabulary by integrating a neural-symbolic prefix tree structure to guide the model's transcription output. To validate our approach, we conducted experiments using a validation dataset comprising maritime data collected within a simulated training environment. A comparison between the original Whisper models of
Authors
(none)
Tags
Stats
Related papers
- Improving Synthetic Data Training For Contextual Biasing Models With A Keyword-aware Cost Function (2025)0.00
- A Multitask Training Approach To Enhance Whisper With Contextual Biasing And Open-vocabulary Keyword Spotting (2023)0.00
- Fine-tuning Whisper On Low-resource Languages For Real-world Applications (2024)0.00
- Whisper-lm: Improving ASR Models With Language Models For Low-resource Languages (2025)3.29
- Robust Acoustic And Semantic Contextual Biasing In Neural Transducers For Speech Recognition (2023)8.60
- Adaptive Contextual Biasing For Transducer Based Streaming Speech Recognition (2023)7.16
- A Whisper Transformer For Audio Captioning Trained With Synthetic Captions And Transfer Learning (2023)0.00
- M2r-whisper: Multi-stage And Multi-scale Retrieval Augmentation For Enhancing Whisper (2024)6.77