DevEval

The 'DevEval' dataset/benchmark is used to evaluate the performance of code generation models by assessing their ability to handle repository-level code with cross-file dependencies and structural context.

Papers using DevEval (7)

Do Not Treat Code as Natural Language: Implications for Repository-Level Code Generation and Beyond2026From Context to Intent: Reasoning-Guided Function-Level Code Completion2025AdaDec: A Uncertainty-Guided Lookahead Decoding Framework for LLM-Based Code Generation2025Can Language Models Replace Programmers for Coding? REPOCOD Says 'Not Yet'2024DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories2024 · 31 citesDevEval: Evaluating Code Generation in Practical Software Projects2024 · 3 citesPrompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study2024 · 3 cites