🧰 Open Source Tools — Awesome Speech Audio

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

Clone a voice in 5 seconds to generate arbitrary speech in real-time

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

Port of OpenAI's Whisper model in C/C++

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

A generative speech model for daily dialogue.

The open-source AI voice studio. Clone, dictate, create.

🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time

Instant voice cloning by MIT and MyShell. Audio foundation model.

Easily train a good VC model with voice data <= 10 mins!

Cross-platform, customizable ML solutions for live and streaming media.

World's first open-source, agentic video production system. 12 pipelines, 52 tools, 500+ agent skills. Turn your AI coding assistant into a full video production studio.

VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning

SOTA Open Source TTS

Deezer source separation library including pretrained models.

SoftVC VITS Singing Voice Conversion

DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

Mastra is the modern TypeScript framework for AI-powered applications and agents.

🚀 AI 全自动短视频引擎 | AI Fully Automated Short Video Engine

Faster Whisper transcription with CTranslate2

🚀 AI 全自动短视频引擎 | AI Fully Automated Short Video Engine

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

Readest is a modern, feature-rich ebook reader designed for avid readers offering seamless cross-platform access, powerful tools, and an intuitive interface to elevate your reading experience.

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System

Buzz transcribes and translates audio offline on your personal computer. Powered by OpenAI's Whisper.

Generate audiobooks from e-books, voice cloning & 1158+ languages!

A TTS model capable of generating ultra-realistic dialogue in one pass.

🌈一个跨平台的划词翻译和OCR软件 | A cross-platform software for text translation and recognition.

Industrial-grade speech recognition toolkit: 170x realtime, 50+ languages, speaker diarization, emotion detection, streaming, and OpenAI-compatible API.

Translate the video from one language to another and embed dubbing & subtitles.

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

Netflix-level subtitle cutting, translation, alignment, and even dubbing - one-click fully automated AI video subtitle team | Netflix级字幕切割、翻译、对齐、甚至加上配音，一键全自动视频搬运AI字幕组

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local processing. no cloud required. Meetily (Meetly Ai - https://meetily.ai) is the #1 Self-hosted, Open-source Ai meeting note taker for macOS & Windows.

🧠 Leon is your open-source personal assistant.

kaldi-asr/kaldi is the official location of the Kaldi project.

Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C# and Node

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V, RK NPU, Axera NPU, Ascend NPU, x86_64 servers, websocket server/client, support 12 programming languages

Lightning-Fast, On-Device, Multilingual TTS — running natively via ONNX.

Drench yourself in Deep Learning, Reinforcement Learning, Machine Learning, Computer Vision, and NLP by learning from these exciting lectures!!

Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text frontend, Speaker Verification System, End-to-End Speech Translation and Keyword Spotting. Won NAACL2022 Best Demo Award.

A PyTorch-based Speech Toolkit

Use Microsoft Edge's online text-to-speech service from Python WITHOUT needing Microsoft Edge or Windows or an API key

A fast, local neural text to speech system

Gradio WebUI for creators and developers, featuring key TTS (Edge-TTS, kokoro) and zero-shot Voice Cloning (E2 & F5-TTS, CosyVoice), with Whisper audio processing, YouTube download, Demucs vocal isolation, and multilingual translation.

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

AI video translation & dubbing tool for humans and AI Agents, powered by LLMs. Full pipeline: download, transcribe, translate, TTS dub, reformat, cover generation. 100+ languages, optimized for YouTube, TikTok, Bilibili, Douyin, and more.AI视频翻译配音工具，面向人类与AI Agent，100+语言全链路，CLI分阶段调用，适配抖音、小红书、哔哩哔哩、视频号、TikTok、YouTube

Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding

Open source real-time translation app for Android that runs locally

:robot: :speech_balloon: Deep learning for Text to Speech (Discussion forum: https://discourse.mozilla.org/c/tts)

AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

End-to-End Speech Processing Toolkit

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

so-vits-svc fork with realtime support, improved interface and more features.

A sound cloning tool with a web interface, using your voice or any sound to record audio / 一个带web界面的声音克隆工具，使用你的音色或任意声音来录制音频