Neural Representations For Modeling Variation In Speech

Abstract

Variation in speech is often quantified by comparing phonetic transcriptions of the same utterance. However, manually transcribing speech is time-consuming and error prone. As an alternative, therefore, we investigate the extraction of acoustic embeddings from several self-supervised neural models. We use these representations to compute word-based pronunciation differences between non-native and native speakers of English, and between Norwegian dialect speakers. For comparison with several earlier studies, we evaluate how well these differences match human perception by comparing them with available human judgements of similarity. We show that speech representations extracted from a specific type of neural model (i.e. Transformers) lead to a better match with human perception than two earlier approaches on the basis of phonetic transcriptions and MFCC-based acoustic features. We furthermore find that features from the neural models can generally best be extracted from one of the middl

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybartelds2020neural

Related papers

• Analyzing Analytical Methods: The Case Of Phonology In Neural Models Of Spoken Language (2020)6.77
• Comparison Of Speech Representations For Automatic Quality Estimation In Multi-speaker Text-to-speech Synthesis (2020)0.00
• How Familiar Does That Sound? Cross-lingual Representational Similarity Analysis Of Acoustic Word Embeddings (2021)0.00
• Speech Representation Analysis Based On Inter- And Intra-model Similarities (2024)2.26
• Analyzing Deep Cnn-based Utterance Embeddings For Acoustic Model Adaptation (2018)6.77
• Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study (2021)4.52
• An Unsupervised Autoregressive Model For Speech Representation Learning (2019)17.26
• Learning Problem-agnostic Speech Representations From Multiple Self-supervised Tasks (2019)15.54