← all papers · overview

Measurement Without Validity: The Compounding Reliability Problem in Agentic AI Evaluation

Abstract

Agentic AI evaluation pipelines produce benchmark scores that justify deployment decisions, safety certifications, and regulatory compliance claims. No formal framework has yet characterized how validity degrades across the stages of these pipelines. We present a three-layer compounding validity model, V_total =0.70; alpha >= 0.67/0.70/0.80 by consequence level) that practitioners and benchmark authors can apply immediately. The framework provides a tractable knowledge-based tool for diagnosing and correcting evaluation pipeline validity before deployment decisions are made.

Related papers

Ranked by semantic similarity — how closely each paper's abstract matches this one (100% = near-identical topic).