ClassEval

ClassEval is a dataset that contains high-quality samples used to evaluate the performance of models on programming tasks, specifically in the context of low-resource general-purpose programming languages.

Papers using ClassEval (12)

ClassEval-T: Evaluating Large Language Models in Class-Level Code Translation2024 · 1 citesCangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language2026Scaling Test-Driven Code Generation from Functions to Classes: An Empirical Study2026Automated Test Suite Enhancement Using Large Language Models with Few-shot Prompting2026From Human to Machine Refactoring: Assessing GPT-4's Impact on Python Class Quality and Readability2026Evaluating Software Process Models for Multi-Agent Class-Level Code Generation2025TALM: Dynamic Tree-Structured Multi-Agent Framework with Long-Term Memory for Scalable Code Generation2025ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation2023 · 21 citesReasoning Runtime Behavior of a Program with LLM: How Far Are We?2024 · 3 citesCoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing2024TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models2024Strategic Optimization and Challenges of Large Language Models in Object-Oriented Programming2024