DevAI

Emerging

2papers using it

2024first seen

DevAI is a benchmark containing 55 realistic automated AI development tasks with rich manual annotations, used to evaluate agentic systems through the Agent-as-a-Judge framework.

🔎 Find this dataset

Papers using DevAI (2)

Multilingual Prompt Localization for Agent-as-a-Judge: Language and Backbone Sensitivity in Requirement-Level Evaluation2026

Agent-as-a-judge: Evaluate Agents With Agents2024 · 3 cites