MultiPL-E

Dataset Card for MultiPL-E Dataset Summary MultiPL-E is a dataset for evaluating large language models for code generation that supports 22 programming languages. It takes the OpenAI HumanEval and the Mostly Basic Python Programs (MBPP) benchmarks and uses little compilers to translate them to other languages. It is ea

Papers using MultiPL-E (14)

SwiftEval: Developing a Language-Specific Benchmark for LLM-generated Code Evaluation2025 · 1 citesAgnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment2025Iterative Self-Training for Code Generation via Reinforced Re-Ranking2025ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation2024Code Llama: Open Foundation Models for Code2023 · 398 citesSantaCoder: don't reach for the stars!2023 · 51 citesA Preliminary Study of Multilingual Code Language Models for Code Generation Task Using Translated Benchmarks2024 · 2 citesInstruction Fusion: Advancing Prompt Evolution through Hybridization2023XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts2024InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct2024$\mathbb{USCD}$: Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decoding2024ExecRepoBench: Multi-level Executable Code Completion Evaluation2024PERC: Plan-As-Query Example Retrieval for Underrepresented Code Generation2024Iterative Self-Training for Code Generation via Reinforced Re-Ranking2025