Verification loops

Probabilistic systems doing deterministic work. The harness intercepts every tool call, validates, executes sandboxed and checks the result: pre-execution before tokens burn, post-execution before anything proceeds.

The escalation gradient: the cost of checking is matched to the risk of being wrong. Level 1 automated, level 2 a second model as judge, level 3 a human hand on the switch. Where the lines are drawn is the harness designer's job.

Check in both directions

Verification loops include all the automated quality checks that validate agent outputs during execution. They are there to ensure that probabilistic systems like agents are doing deterministic work. Rather than relying on the model to be correct, the harness intercepts a tool call, validates its parameters, executes in a sandbox, cleans the output, and injects the result back into context. Verification happens in both directions. Pre-execution, where hallucinated function calls are caught along with invalid parameter types and references to nonexistent APIs, avoiding tokens spent on failed retries. And post-execution, during which the harness validates that what the agent did actually worked: automated testing for code, visual verification for UI changes.

The loop prevents a documented family of failures: premature termination of an incomplete task; infinite refinement, where the agent endlessly "improves" its output while consuming compute; reasoning-action mismatch, roughly 14% of agent failures in the MAST taxonomy, where the agent reasons one thing but does another; step repetition; and hallucination cascades, where fabricated information in one step corrupts every downstream decision.

Mistake-proofing the environment

The design philosophy is to make incorrect behavior mechanically impossible rather than asking the model to be correct. If an agent makes a mistake the engineered environment is updated so that the mistake cannot physically recur, a principle championed by Mitchell Hashimoto. Updated instruction files, tools that force verification, staged checkpoints with automated checks.

Drawing the lines

Not every verification is equal; the escalation gradient at the level above shows the three tiers. How the gradient is used matters: place everything under level 3 and the agent is an expensive approval queue; leave everything at level 1 and mistakes eventually reach the real world. The harness designer's job is to draw the line between what is routine, what needs a second opinion and what needs a human hand on the switch.