Openace: An Open Benchmark For Evaluating Audio Coding Performance

Abstract

Audio and speech coding lack unified evaluation and open-source testing. Many candidate systems were evaluated on proprietary, non-reproducible, or small data, and machine learning-based codecs are often tested on datasets with similar distributions as trained on, which is unfairly compared to digital signal processing-based codecs that usually work well with unseen data. This paper presents a full-band audio and speech coding quality benchmark with more variable content types, including traditional open test vectors. An example use case of audio coding quality assessment is presented with open-source Opus, 3GPP's EVS, and recent ETSI's LC3 with LC3+ used in Bluetooth LE Audio profiles. Besides, quality variations of emotional speech encoding at 16 kbps are shown. The proposed open-source benchmark contributes to audio and speech coding democratization and is available at https://github.com/JozefColdenhoff/OpenACE.

Stats

• citations0
• S2 citations—
• github stars11
• HF likes0
• heat score2.16
• arxiv keycoldenhoff2024openace

Code

• JozefColdenhoff/OpenACE—

Related papers

• Codec-superb @ SLT 2024: A Lightweight Benchmark For Neural Audio Codec Models (2024)7.16
• Towards Evaluating Generative Audio: Insights From Neural Audio Codec Embedding Distances (2025)0.00
• Espnet-codec: Comprehensive Training And Evaluation Of Neural Codecs For Audio, Music, And Speech (2024)9.03
• Audiobench: A Universal Benchmark For Audio Large Language Models (2024)10.21
• Open ASR Leaderboard: Towards Reproducible And Transparent Multilingual And Long-form Speech Recognition Evaluation (2025)0.00
• Speech Quality Factors For Traditional And Neural-based Low Bit Rate Vocoders (2020)7.16
• Neural Speech And Audio Coding: Modern AI Technology Meets Traditional Codecs (2024)7.16
• Visqol V3: An Open Source Production Ready Objective Speech And Audio Metric (2020)15.83