Who&When

The 'Who&When' dataset/benchmark contains data used to evaluate the step-level accuracy of causal attribution in LLM agents, specifically focusing on identifying which step in an agent's decision-making process caused a failure.

Papers using Who&When (6)

FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search2026StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems2026Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures2026SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation2026Which Agent Causes Task Failures And When? On Automated Failure Attribution Of LLM Multi-agent Systems2025VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems2026