ClassEval

Dataset Card for FudanSELab ClassEval Dataset Summary We manually build ClassEval of 100 class-level Python coding tasks, consists of 100 classes and 412 methods, and average 33.1 test cases per class. For 100 class-level tasks, diversity is maintained by encompassing these tasks over a wide spectrum of topics, includi

Papers using ClassEval (12)

ClassEval-T: Evaluating Large Language Models in Class-Level Code Translation2024 · 1 citesCangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language2026Scaling Test-Driven Code Generation from Functions to Classes: An Empirical Study2026Automated Test Suite Enhancement Using Large Language Models with Few-shot Prompting2026From Human to Machine Refactoring: Assessing GPT-4's Impact on Python Class Quality and Readability2026Evaluating Software Process Models for Multi-Agent Class-Level Code Generation2025TALM: Dynamic Tree-Structured Multi-Agent Framework with Long-Term Memory for Scalable Code Generation2025ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation2023 · 25 citesReasoning Runtime Behavior of a Program with LLM: How Far Are We?2024 · 3 citesCoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing2024TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models2024Strategic Optimization and Challenges of Large Language Models in Object-Oriented Programming2024