SIMMER

SIMMER is a benchmark that evaluates latent failures in large language model (LLM) planning using a human-curated symbolic world model focused on the kitchen domain, which includes 77 actions, 262 unique objects, and approximately 46,800 possible interactions derived from real-world cooking scripts.

Papers using SIMMER (2)

SIMMER: Benchmarking Latent Failures in LLM Executable Planning with a World Model2026Debate2Create: Robot Co-design via Multi-Agent LLM Debate2025