SGEM: Test-time Adaptation For Automatic Speech Recognition Via Sequential-level Generalized Entropy Minimization

Abstract

Automatic speech recognition (ASR) models are frequently exposed to data distribution shifts in many real-world scenarios, leading to erroneous predictions. To tackle this issue, an existing test-time adaptation (TTA) method has recently been proposed to adapt the pre-trained ASR model on unlabeled test instances without source data. Despite decent performance gain, this work relies solely on naive greedy decoding and performs adaptation across timesteps at a frame level, which may not be optimal given the sequential nature of the model output. Motivated by this, we propose a novel TTA framework, dubbed SGEM, for general ASR models. To treat the sequential output, SGEM first exploits beam search to explore candidate output logits and selects the most plausible one. Then, it utilizes generalized entropy minimization and negative sampling as unsupervised objectives to adapt the model. SGEM achieves state-of-the-art performance for three mainstream ASR models under various domain shifts.

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keykim2023sgem

Related papers

• Examining Test-time Adaptation For Personalized Child Speech Recognition (2024)0.00
• EMO-TTA: Improving Test-time Adaptation Of Audio-language Models For Speech Emotion Recognition (2025)0.00
• SLM-TTA: A Framework For Test-time Adaptation Of Generative Spoken Language Models (2025)0.00
• Continual Test-time Adaptation For End-to-end Speech Recognition On Noisy Speech (2024)4.52
• Continual Learning In Machine Speech Chain Using Gradient Episodic Memory (2024)2.26
• Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation For Automatic Speech Recognition (2022)8.09
• LI-TTA: Language Informed Test-time Adaptation For Automatic Speech Recognition (2024)3.58
• Advancing Test-time Adaptation In Wild Acoustic Test Settings (2023)2.26