Investigation Of Speaker-adaptation Methods In Transformer Based ASR

Abstract

End-to-end models are fast replacing the conventional hybrid models in automatic speech recognition. Transformer, a sequence-to-sequence model, based on self-attention popularly used in machine translation tasks, has given promising results when used for automatic speech recognition. This paper explores different ways of incorporating speaker information at the encoder input while training a transformer-based model to improve its speech recognition performance. We present speaker information in the form of speaker embeddings for each of the speakers. We experiment using two types of speaker embeddings: x-vectors and novel s-vectors proposed in our previous work. We report results on two datasets a) NPTEL lecture database and b) Librispeech 500-hour split. NPTEL is an open-source e-learning portal providing lectures from top Indian universities. We obtain improvements in the word error rate over the baseline through our approach of integrating speaker embeddings into the model.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyshetty2020investigation

Related papers

• Speaker Adaptation For End-to-end Speech Recognition Systems In Noisy Environments (2022)0.00
• Adapting End-to-end Neural Speaker Verification To New Languages And Recording Conditions With Adversarial Training (2018)9.59
• Investigation Of Speaker Representation For Target-speaker Speech Processing (2024)4.52
• S-vectors And TESA: Speaker Embeddings And A Speaker Authenticator Based On Transformer Encoder (2020)11.08
• Improving Transformer-based Networks With Locality For Automatic Speaker Verification (2023)0.00
• Vae-based Domain Adaptation For Speaker Verification (2019)7.50
• Efficient Adapter Tuning Of Pre-trained Speech Models For Automatic Speaker Verification (2024)0.00
• Adapting Self-supervised Models To Multi-talker Speech Recognition Using Speaker Embeddings (2022)10.61