Retrieving Speaker Information From Personalized Acoustic Models For Speech Recognition

Abstract

The widespread of powerful personal devices capable of collecting voice of their users has opened the opportunity to build speaker adapted speech recognition system (ASR) or to participate to collaborative learning of ASR. In both cases, personalized acoustic models (AM), i.e. fine-tuned AM with specific speaker data, can be built. A question that naturally arises is whether the dissemination of personalized acoustic models can leak personal information. In this paper, we show that it is possible to retrieve the gender of the speaker, but also his identity, by just exploiting the weight matrix changes of a neural acoustic model locally adapted to this speaker. Incidentally we observe phenomena that may be useful towards explainability of deep neural networks in the context of speech processing. Gender can be identified almost surely using only the first layers and speaker verification performs well when using middle-up layers. Our experimental study on the TED-LIUM 3 dataset with HMM/T

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keymdhaffar2021retrieving

Related papers

• Privacy Attacks For Automatic Speech Recognition Acoustic Models In A Federated Learning Framework (2021)9.23
• On-device Speaker Anonymization Of Acoustic Embeddings For ASR Based Onflexible Location Gradient Reversal Layer (2023)0.00
• Personalized Acoustic Modeling By Weakly Supervised Multi-task Deep Learning Using Acoustic Tokens Discovered From Unlabeled Data (2017)2.26
• Speaker De-identification System Using Autoencoders And Adversarial Training (2020)0.00
• Speaker Anonymization Using X-vector And Neural Waveform Models (2019)0.00
• Language-independent Speaker Anonymization Approach Using Self-supervised Pre-trained Models (2022)9.92
• A Highly Adaptive Acoustic Model For Accurate Multi-dialect Speech Recognition (2022)10.85
• Deep Learning For Speaker Identification: Architectural Insights From AB-1 Corpus Analysis And Performance Evaluation (2024)0.00