Cloud Coding Agent

A cloud coding agent that never grades its own homework: a separate, read-only verifier tests the running app and shows evidence for every check.

Solo, on top of an open-source agent scaffold · Jul 2026

Results

• 1 real false pass caught by the eval on its first run
• 8 / 8 verdicts correct after the fix
• 0 false passes on the re-run
• 1 day to build the verify loop and eval

The problem

Coding agents often say a task is done when the app does not actually work, so people end up re-checking everything themselves. Trust needs proof the model cannot fake, not a confident summary.

What I built

• A turn pipeline: a fresh model call writes a browser-checkable definition of done, the builder works in an E2B cloud sandbox, then a verifier with fresh context and read-only tools tests the running app.
• Trust lives in the harness, not the model: screenshots are captured server-side at the moment of each assertion, verdicts are computed by code, and 'unable to verify' is an honest outcome.
• Fix and re-verify loops: a fix gets the failing checks plus the verifier's evidence, then every item is re-checked, because fixes can break things that passed before.
• A small eval that grades the verifier itself, with ground-truth HTTP probes, repeated trials, a no-verify baseline, and fault injection.

The hard part

Making a 'verified' badge mean something when the verifier is itself a model. I moved every trust-bearing output into code, then built an eval that fact-checks the verifier with HTTP probes independent of both agents. Its first full run caught a false pass: after the app server was killed, the browser cache still served the page, so the verifier saw a working app that was not there. I fixed it mechanically with cache-busting navigation, and the re-run scored 8 of 8 verdicts correct with zero false passes.

How it fits together

• Chat UI to Turn orchestrator (task)
• Turn orchestrator to Definition of done (checklist)
• Turn orchestrator to Builder agent (build)
• Turn orchestrator to Verifier agent (verify)
• Builder agent to Running app (write, run)
• Verifier agent to Running app (browser checks)
• Turn orchestrator to Evidence + traces (verdicts)

Tradeoff: verification makes every turn slower, but a pass now means something because no model writes its own verdict.

Decisions

• Verdicts are computed by code from recorded checks, not written by a model, so a model can't talk its way into a pass.
• The verifier starts with fresh context and read-only tools instead of sharing the builder's, so it can't inherit the builder's assumptions or quietly fix things.
• 'Unable to verify' is a real outcome, not a forced pass or fail, so an honest gap is never dressed up as a result.

Tech: TypeScript, Agents, Evals, Sandboxes.

By David Antwi (david.antwi@yale.edu). Markdown version: /work/cloud-coding-agent.md. Full profile: /llms.txt.