GAIA

GAIA dataset GAIA is a benchmark which aims at evaluating next-generation LLMs (LLMs with augmented capabilities due to added tooling, efficient prompting, access to search, etc). We added gating to prevent bots from scraping the dataset. Please do not reshare the validation or test set in a crawlable format. Data and

Papers using GAIA (54)

EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments2026AWorld: Orchestrating the Training Recipe for Agentic AI2025 · 11 citesRethinking Memory as Continuously Evolving Connectivity2026HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry2026PACE: A Proxy for Agentic Capability Evaluation2026SlimSearcher: Training Efficiency-Aware Web Agents via Adaptive Reward Gating2026Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window2025CodeAgents: A Token-Efficient Framework for Codified Multi-Agent Reasoning in LLMs2025 · 4 citesFrom Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws2026SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference2026Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability2026Characterization of Multi-Model Agentic AI Systems on General Tasks via Trace-Driven Simulation2026ActiveMem: Distributed Active Memory for Long-Horizon LLM Reasoning2026Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning2026LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception2026ReGRPO: Reflection-Augmented Policy Optimization for Tool-Using Agents2026DecisionBench: A Benchmark for Emergent Delegation in Long-Horizon Agentic Workflows2026Where LLM Agents Fail And How They Can Learn From Failures2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation2025 · 1 citesDeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment2026Infantagent-next: A Multimodal Generalist Agent For Automated Computer Interaction2025ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability2026Maximizing Rollout Informativeness under a Fixed Budget: A Submodular View of Tree Search for Tool-Use Agentic Reinforcement Learning2026Gaia-v2-lilt: Multilingual Adaptation Of Agent Benchmark Beyond Translation2026EvoMAS: Learning Execution-Time Workflows for Multi-Agent Systems2026PIVOT: Bridging Planning and Execution in LLM Agents via Trajectory Refinement2026OpenJarvis: Personal AI, On Personal Devices2026LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent2026AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration2026ReThinker: Scientific Reasoning by Rethinking with Guided Reflection and Confidence Control2026Learning to Share: Selective Memory for Efficient Parallel Agentic Systems2026Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization2026MiroFlow: Towards High-Performance and Robust Open-Source Agent Framework for General Deep Research Tasks2026WebAnchor: Anchoring Agent Planning to Stabilize Long-Horizon Web Reasoning2026Beyond Rule-Based Workflows: An Information-Flow-Orchestrated Multi-Agents Paradigm via Agent-to-Agent Communication from CORAL2026Yunque DeepResearch Technical Report2026MonoScale: Scaling Multi-Agent System with Monotonic Improvement2026Evoroute: Experience-driven Self-routing LLM Agent Systems2026Unifying Dynamic Tool Creation and Cross-Task Experience Sharing through Cognitive Memory Architecture2025Mirothinker: Pushing The Performance Boundaries Of Open-source Research Agents Via Model, Context, And Interactive Scaling2025FlowSearch: Advancing deep research with dynamic structured knowledge flow2025MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning2025ProtocolBench: Which LLM MultiAgent Protocol to Choose?2025Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use2025Gradientsys: A Multi-Agent LLM Scheduler with ReAct Orchestration2025Auto-eval Judge: Towards A General Agentic Framework For Task Completion Evaluation2025COMPASS: Enhancing Agent Long-horizon Reasoning With Evolving Context2025Efficient Agents: Building Effective Agents While Reducing Cost2025Researstudio: A Human-intervenable Framework For Building Controllable Deep-research Agents2025Divide, Optimize, Merge: Fine-Grained LLM Agent Optimization at Scale2025Affordable AI Assistants with Knowledge Graph of Thoughts2025Multi-modal Agent Tuning: Building A Vlm-driven Agent For Efficient Tool Usage2024