Advancing Singlish Understanding: Bridging The Gap With Datasets And Multimodal Models

Abstract

Singlish, a Creole language rooted in English, is a key focus in linguistic research within multilingual and multicultural contexts. However, its spoken form remains underexplored, limiting insights into its linguistic structure and applications. To address this gap, we standardize and annotate the largest spoken Singlish corpus, introducing the Multitask National Speech Corpus (MNSC). These datasets support diverse tasks, including Automatic Speech Recognition (ASR), Spoken Question Answering (SQA), Spoken Dialogue Summarization (SDS), and Paralinguistic Question Answering (PQA). We release standardized splits and a human-verified test set to facilitate further research. Additionally, we propose SingAudioLLM, a multi-task multimodal model leveraging multimodal large language models to handle these tasks concurrently. Experiments reveal our models adaptability to Singlish context, achieving state-of-the-art performance and outperforming prior models by 10-30% in comparison with other A

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2025advancing

Related papers

• Meralion-audiollm: Bridging Audio And Language With Large Language Models (2024)0.00
• Meralion-speechencoder: Towards A Speech Foundation Model For Singapore And Beyond (2024)0.00
• MMSU: A Massive Multi-task Spoken Language Understanding And Reasoning Benchmark (2025)2.29
• Merlion CCS Challenge: A English-mandarin Code-switching Child-directed Speech Corpus For Language Identification And Diarization (2023)0.00
• Exploring Fine-tuning Of Large Audio Language Models For Spoken Language Understanding Under Limited Speech Data (2025)0.00
• Mmmmodal -- Multi-images Multi-audio Multi-turn Multi-modal (2024)0.00
• Audiosetmix: Enhancing Audio-language Datasets With Llm-assisted Augmentations (2024)0.00
• MCAT: Scaling Many-to-many Speech-to-text Translation With Mllms To 70 Languages (2025)2.41