Star Temporal Classification: Sequence Classification With Partially Labeled Data

Abstract

We develop an algorithm which can learn from partially labeled and unsegmented sequential data. Most sequential loss functions, such as Connectionist Temporal Classification (CTC), break down when many labels are missing. We address this problem with Star Temporal Classification (STC) which uses a special star token to allow alignments which include all possible tokens whenever a token could be missing. We express STC as the composition of weighted finite-state transducers (WFSTs) and use GTN (a framework for automatic differentiation with WFSTs) to compute gradients. We perform extensive experiments on automatic speech recognition. These experiments show that STC can recover most of the performance of supervised baseline when up to 70% of the labels are missing. We also perform experiments in handwriting recognition to show that our method easily applies to other sequence classification tasks.

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keypratap2022star

Related papers

• Semi-supervised Speech Recognition Via Graph-based Temporal Classification (2020)9.59
• Training LDCRF Model On Unsegmented Sequences Using Connectionist Temporal Classification (2016)2.26
• Gram-ctc: Automatic Unit Selection And Target Decomposition For Sequence Labelling (2017)0.00
• Variational Connectionist Temporal Classification For Order-preserving Sequence Modeling (2023)5.24
• Softctc -- Semi-supervised Learning For Text Recognition Using Soft Pseudo-labels (2022)5.24
• Multitask Learning With CTC And Segmental CRF For Speech Recognition (2017)0.00
• Bypass Temporal Classification: Weakly Supervised Automatic Speech Recognition With Imperfect Transcripts (2023)7.50
• Self-attention Networks For Connectionist Temporal Classification In Speech Recognition (2019)14.55