Leverage Unlabeled Data For Abstractive Speech Summarization With Self-supervised Learning And Back-summarization
2020 · Paul Tardy, Louis de Seynes, François Hernandez, et al.
Abstract
Supervised approaches for Neural Abstractive Summarization require large annotated corpora that are costly to build. We present a French meeting summarization task where reports are predicted based on the automatic transcription of the meeting audio recordings. In order to build a corpus for this task, it is necessary to obtain the (automatic or manual) transcription of each meeting, and then to segment and align it with the corresponding manual report to produce training examples suitable for training. On the other hand, we have access to a very large amount of unaligned data, in particular reports without corresponding transcription. Reports are professionally written and well formatted making pre-processing straightforward. In this context, we study how to take advantage of this massive amount of unaligned data using two approaches (i) self-supervised pre-training using a target-side denoising encoder-decoder model; (ii) back-summarization i.e. reversing the summarization process by
Authors
(none)
Tags
Stats
Related papers
- Speech Summarization Using Restricted Self-attention (2021)0.00
- Augsumm: Towards Generalizable Speech Summarization Using Synthetic Labels From Large Language Model (2024)4.53
- Transfer Learning From Pre-trained Language Models Improves End-to-end Speech Summarization (2023)6.77
- Team MTS @ Automin 2021: An Overview Of Existing Summarization Approaches And Comparison To Unsupervised Summarization Techniques (2024)0.00
- Sentence-wise Speech Summarization: Task, Datasets, And End-to-end Modeling With LM Knowledge Distillation (2024)5.84
- Prompting Large Language Models With Audio For General-purpose Speech Summarization (2024)6.34
- Almost Unsupervised Text To Speech And Automatic Speech Recognition (2019)0.00
- Semantic Enrichment Towards Efficient Speech Representations (2023)0.00