CRUXEval

CRUXEval: Code Reasoning, Understanding, and Execution Evaluation 🏠 Home Page • 💻 GitHub Repository • 🏆 Leaderboard • 🔎 Sample Explorer CRUXEval (Code Reasoning, Understanding, and eXecution Evaluation) is a benchmark of 800 Python functions and input-output pairs. The benchmark consists of two tasks, CRUXEval-I (i

Papers using CRUXEval (10)

SpecEval: Evaluating Code Comprehension in Large Language Models via Program Specifications2024 · 1 citesHow Robustly do LLMs Understand Execution Semantics?2026Towards a Neural Debugger for Python2026STEPWISE-CODEX-Bench: Evaluating Complex Multi-Function Comprehension and Fine-Grained Execution Reasoning2025Are Large Language Models Robust in Understanding Code Against Semantics-Preserving Mutations?2025What I cannot execute, I do not understand: Training and Evaluating LLMs on Program Execution Traces2025A Tool for In-depth Analysis of Code Execution Reasoning of Large Language Models2025CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution2024 · 5 citesCRUXEval: A Benchmark for Code Reasoning, Understanding and Execution2024Towards a Neural Debugger for Python2026