Training Speaker Embedding Extractors Using Multi-speaker Audio With Unknown Speaker Boundaries

Abstract

In this paper, we demonstrate a method for training speaker embedding extractors using weak annotation. More specifically, we are using the full VoxCeleb recordings and the name of the celebrities appearing on each video without knowledge of the time intervals the celebrities appear in the video. We show that by combining a baseline speaker diarization algorithm that requires no training or parameter tuning, a modified loss with aggregation over segments, and a two-stage training approach, we are able to train a competitive ResNet-based embedding extractor. Finally, we experiment with two different aggregation functions and analyze their behaviour in terms of their gradients.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keystafylakis2022training

Related papers

• State-of-the-art Embeddings With Video-free Segmentation Of The Source Voxceleb Data (2024)0.00
• Weakly Supervised Training Of Speaker Identification Models (2018)5.84
• Recursive Attentive Pooling For Extracting Speaker Embeddings From Multi-speaker Recordings (2024)2.26
• Leveraging Speaker Attribute Information Using Multi Task Learning For Speaker Verification And Diarization (2020)6.34
• ECAPA2: A Hybrid Neural Network Architecture And Training Strategy For Robust Speaker Embeddings (2024)0.00
• Unified Hypersphere Embedding For Speaker Recognition (2018)0.00
• Self-supervised Training Of Speaker Encoder With Multi-modal Diverse Positive Pairs (2022)8.35
• Single-channel Speech Separation With Auxiliary Speaker Embeddings (2019)0.00