Similarity Analysis Of Self-supervised Speech Representations

Abstract

Self-supervised speech representation learning has recently been a prosperous research topic. Many algorithms have been proposed for learning useful representations from large-scale unlabeled data, and their applications to a wide range of speech tasks have also been investigated. However, there has been little research focusing on understanding the properties of existing approaches. In this work, we aim to provide a comparative study of some of the most representative self-supervised algorithms. Specifically, we quantify the similarities between different self-supervised representations using existing similarity measures. We also design probing tasks to study the correlation between the models' pre-training loss and the amount of specific speech information contained in their learned representations. In addition to showing how various self-supervised models behave differently given the same input, our study also finds that the training objective has a higher impact on representation s

Stats

• citations21
• S2 citations—
• github stars0
• HF likes0
• heat score10.07
• arxiv keychung2020similarity

Related papers

• Revisiting Self-supervised Learning Of Speech Representation From A Mutual Information Perspective (2024)4.52
• Speech Representation Analysis Based On Inter- And Intra-model Similarities (2024)2.26
• Learning Problem-agnostic Speech Representations From Multiple Self-supervised Tasks (2019)15.54
• Comparative Layer-wise Analysis Of Self-supervised Speech Models (2022)0.00
• Layer-wise Analysis Of A Self-supervised Speech Representation Model (2021)17.07
• An Exploration Of Self-supervised Pretrained Representations For End-to-end Speech Recognition (2021)12.25
• Phone And Speaker Spatial Organization In Self-supervised Speech Representations (2023)2.26
• Automatic Data Augmentation Selection And Parametrization In Contrastive Self-supervised Speech Representation Learning (2022)5.24