Experimenting With Additive Margins For Contrastive Self-supervised Speaker Verification

Abstract

Most state-of-the-art self-supervised speaker verification systems rely on a contrastive-based objective function to learn speaker representations from unlabeled speech data. We explore different ways to improve the performance of these methods by: (1) revisiting how positive and negative pairs are sampled through a "symmetric" formulation of the contrastive loss; (2) introducing margins similar to AM-Softmax and AAM-Softmax that have been widely adopted in the supervised setting. We demonstrate the effectiveness of the symmetric contrastive loss which provides more supervision for the self-supervised task. Moreover, we show that Additive Margin and Additive Angular Margin allow reducing the overall number of false negatives and false positives by improving speaker separability. Finally, by combining both techniques and training a larger model we achieve 7.50% EER and 0.5804 minDCF on the VoxCeleb1 test set, which outperforms other contrastive self supervised methods on speaker verific

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keylepage2023experimenting

Related papers

• Additive Margin In Contrastive Self-supervised Frameworks To Learn Discriminative Speaker Representations (2024)2.26
• Label-efficient Self-supervised Speaker Verification With Information Maximization And Contrastive Learning (2022)6.77
• Discriminative Speaker Representation Via Contrastive Learning With Class-aware Attention In Angular Space (2022)8.60
• Asymmetric Clean Segments-guided Self-supervised Learning For Robust Speaker Verification (2023)5.84
• Large Margin Softmax Loss For Speaker Verification (2019)14.66
• Self-supervised Text-independent Speaker Verification Using Prototypical Momentum Contrastive Learning (2020)12.93
• Speaker Representation Learning Via Contrastive Loss With Maximal Speaker Separability (2022)10.68
• Angular Softmax Loss For End-to-end Speaker Verification (2018)11.19