Exploring Wav2vec 2.0 On Speaker Verification And Language Identification

Abstract

Wav2vec 2.0 is a recently proposed self-supervised framework for speech representation learning. It follows a two-stage training process of pre-training and fine-tuning, and performs well in speech recognition tasks especially ultra-low resource cases. In this work, we attempt to extend self-supervised framework to speaker verification and language identification. First, we use some preliminary experiments to indicate that wav2vec 2.0 can capture the information about the speaker and language. Then we demonstrate the effectiveness of wav2vec 2.0 on the two tasks respectively. For speaker verification, we obtain a new state-of-the-art result, Equal Error Rate (EER) of 3.61% on the VoxCeleb1 dataset. For language identification, we obtain an EER of 12.02% on 1 second condition and an EER of 3.47% on full-length condition of the AP17-OLR dataset. Finally, we utilize one model to achieve the unified modeling by the multi-task learning for the two tasks.

Stats

• citations119
• S2 citations—
• github stars0
• HF likes0
• heat score15.59
• arxiv keyfan2020exploring

Related papers

• Fine-tuning Wav2vec2 For Speaker Recognition (2021)18.88
• Multitask Detection Of Speaker Changes, Overlapping Speech And Voice Activity Using Wav2vec 2.0 (2022)11.86
• Multi-task Voice Activated Framework Using Self-supervised Learning (2021)6.34
• A Noise-robust Self-supervised Pre-training Model Based Speech Representation Learning For Automatic Speech Recognition (2022)11.19
• Wav2vec 2.0: A Framework For Self-supervised Learning Of Speech Representations (2020)0.00
• Improved Language Identification Through Cross-lingual Self-supervised Learning (2021)10.61
• Unsupervised Speech Intelligibility Assessment With Utterance Level Alignment Distance Between Teacher And Learner Wav2vec-2.0 Representations (2023)0.00
• Enhancing Speaker Verification With W2v-bert 2.0 And Knowledge Distillation Guided Structured Pruning (2025)3.33