Learning Word Embeddings From Speech

Abstract

In this paper, we propose a novel deep neural network architecture, Sequence-to-Sequence Audio2Vec, for unsupervised learning of fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the segments, and are close to other vectors in the embedding space if their corresponding segments are semantically similar. The design of the proposed model is based on the RNN Encoder-Decoder framework, and borrows the methodology of continuous skip-grams for training. The learned vector representations are evaluated on 13 widely used word similarity benchmarks, and achieved competitive results to that of GloVe. The biggest advantage of the proposed model is its capability of extracting semantic information of audio segments taken directly from raw speech, without relying on any other modalities such as text or images, which are challenging and expensive to collect and annotate.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychung2017learning

Related papers

• Speech2vec: A Sequence-to-sequence Framework For Learning Word Embeddings From Speech (2018)14.15
• Audio Word2vec: Unsupervised Learning Of Audio Segment Representations Using Sequence-to-sequence Autoencoder (2016)0.00
• Segmental Audio Word2vec: Representing Utterances As Sequences Of Vectors With Applications In Spoken Term Detection (2018)11.08
• Language Transfer Of Audio Word2vec: Learning Audio Segment Representations Without Target Language Data (2017)3.58
• Spoken Word2vec: Learning Skipgram Embeddings From Speech (2023)3.58
• Discriminative Acoustic Word Embeddings: Recurrent Neural Network-based Approaches (2016)0.00
• Improved Audio Embeddings By Adjacency-based Clustering With Applications In Spoken Term Detection (2018)0.00
• Acoustic Neighbor Embeddings (2020)0.00