Speech2vec: A Sequence-to-sequence Framework For Learning Word Embeddings From Speech

Abstract

In this paper, we propose a novel deep neural network architecture, Speech2Vec, for learning fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the underlying spoken words, and are close to other vectors in the embedding space if their corresponding underlying spoken words are semantically similar. The proposed model can be viewed as a speech version of Word2Vec. Its design is based on a RNN Encoder-Decoder framework, and borrows the methodology of skipgrams or continuous bag-of-words for training. Learning word embeddings directly from speech enables Speech2Vec to make use of the semantic information carried by speech that does not exist in plain text. The learned word embeddings are evaluated and analyzed on 13 widely used word similarity benchmarks, and outperform word embeddings learned by Word2Vec from the transcriptions.

Stats

• citations76
• S2 citations—
• github stars0
• HF likes0
• heat score14.15
• arxiv keychung2018speech2vec

Related papers

• Learning Word Embeddings From Speech (2017)0.00
• Spoken Word2vec: Learning Skipgram Embeddings From Speech (2023)3.58
• Audio Word2vec: Unsupervised Learning Of Audio Segment Representations Using Sequence-to-sequence Autoencoder (2016)0.00
• Segmental Audio Word2vec: Representing Utterances As Sequences Of Vectors With Applications In Spoken Term Detection (2018)11.08
• Phonetic-and-semantic Embedding Of Spoken Words With Applications In Spoken Content Retrieval (2018)9.76
• Language Transfer Of Audio Word2vec: Learning Audio Segment Representations Without Target Language Data (2017)3.58
• Wavthruvec: Latent Speech Representation As Intermediate Features For Neural Speech Synthesis (2022)10.07
• Data2vec: A General Framework For Self-supervised Learning In Speech, Vision And Language (2022)0.00