HumanEval
Emerging27papers using it
2024first seen
HumanEval-X is a benchmark for the evaluation of the multilingual ability of code generative models. It consists of 820 high-quality human-crafted data samples (each with test cases) in Python, C++, Java, JavaScript, and Go, and can be used for various tasks.
Papers using HumanEval (15)
- Smarter Saboteurs, Better Fixers: Scaling & Security in Linear Multi-Agent WorkflowsHonest Lying: Understanding Memory Confabulation in Reflexive AgentsG-Designer: Architecting Multi-agent Communication Topologies via Graph Neural NetworksSafesieve: From Heuristics To Experience In Progressive Pruning For Llm-based Multi-agent CommunicationOmniCode: A Benchmark for Evaluating Software Engineering AgentsTDPGen: Optimizing Agentic Code Generation via Test-Driven Planning and Hierarchical ReActR2V Agent: Teaching SLMs When to Ask for HelpGraph-of-Agents: A Graph-based Framework for Multi-Agent LLM CollaborationCARD: Towards Conditional Design of Multi-agent Topological StructuresAdaptive Confidence Gating In Multi-agent Collaboration For Efficient And Optimized Code GenerationMAR:Multi-Agent Reflexion Improves Reasoning Abilities in LLMsA Multi-Agent Framework for Stateful Inference-Time SearchAgentGroupChat-V2: Divide-and-Conquer Is What LLM-Based Multi-Agent System NeedNexus: A Lightweight and Scalable Multi-Agent Framework for Complex Tasks AutomationGuided Code Generation with LLMs: A Multi-Agent Framework for Complex Code Tasks