Spoken Word2vec: Learning Skipgram Embeddings From Speech

Abstract

Text word embeddings that encode distributional semantics work by modeling contextual similarities of frequently occurring words. Acoustic word embeddings, on the other hand, typically encode low-level phonetic similarities. Semantic embeddings for spoken words have been previously explored using analogous algorithms to Word2Vec, but the resulting vectors still mainly encoded phonetic rather than semantic features. In this paper, we examine the assumptions and architectures used in previous works and show experimentally how shallow skipgram-like algorithms fail to encode distributional semantics when the input units are acoustically correlated. We illustrate the potential of an alternative deep end-to-end variant of the model and examine the effects on the resulting embeddings, showing positive results of semantic relatedness in the embedding space.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keysayeed2023spoken

Related papers

• Speech2vec: A Sequence-to-sequence Framework For Learning Word Embeddings From Speech (2018)14.15
• Learning Word Embeddings From Speech (2017)0.00
• Phonetic-and-semantic Embedding Of Spoken Words With Applications In Spoken Content Retrieval (2018)9.76
• Segmental Audio Word2vec: Representing Utterances As Sequences Of Vectors With Applications In Spoken Term Detection (2018)11.08
• On The Difficulty Of A Distributional Semantics Of Spoken Language (2018)0.00
• Bootstrapping Meaning Through Listening: Unsupervised Learning Of Spoken Sentence Embeddings (2022)2.26
• Improved Audio Embeddings By Adjacency-based Clustering With Applications In Spoken Term Detection (2018)0.00
• Exploiting Sentence And Context Representations In Deep Neural Models For Spoken Language Understanding (2016)0.00