Review and Tests: How the Engine Checks Itself

In short

The engine doesn't trust its own "looks fine" and verifies itself through two independent mechanisms. The first is prisms: angles of independent review, each judging on its own without seeing anyone else's verdict; five prisms hold a veto with weight 2 (security, tenancy, gating, regression, data-integrity), each with its own blockIf red lines and triggers by phase, artifact and stack (a11y only switches on when there's UI, data-integrity only on migrations). The second is adversarial verify: at least two independent votes of "done". Both mechanisms are exported into the package as a literal recipe so they work even on a weak target model. The "review to ideal" and provable-testing layers are available at the builder level — this article covers those too.

What it is

The engine doesn't trust its own "looks fine". It verifies itself through two independent mechanisms: prisms (multi-angle review of an artifact) and testing (QA with provable tests). Both are exported into the package as a literal recipe so they work even on a weak model.

Prisms — independent critics

A prism is an angle of independent assessment taken from a role's point of view, NOT an executing agent. The contract: an independent verdict of pass | warn | block plus findings with severity. The recipe in the package demands this outright:

Run the artifact through the angles INDEPENDENTLY: each angle judges on its own and does not see the others' verdicts.

The catalog holds 13 prisms. Five of them are veto prisms (invariants promoted to the rank of judge): security (veto, weight 2), tenancy (veto, weight 2), gating (veto, weight 2), regression (veto, weight 2), data-integrity (veto, weight 2). Every prism has a blockIf — its red lines — and a trigger defining which phases, artifacts and stacks switch it on (a11y only when there's UI, data-integrity only on migrations).

Verdict synthesis (the deterministic rule from orchestration.synthesize):

Any critical from a veto prism → BLOCK; otherwise BLOCK if the summed weight of block votes ≥ the threshold; a tie → BLOCK (safety wins).

In other words, one security prism raising a critical outweighs any number of "all good" votes from the rest.

Orchestration: adversarial verify, ≥2 votes

On top of the prisms sits the vote rule (verifyVotes, set to 2 in Quest):

BEFORE handing over a step, collect at least 2 independent votes of "done": the first is your implementation; the rest are skeptical reviewers who LOOK for what will break and where things were glossed over. A single "looks fine" is NOT done.

Plus fan-out — "one writes, ANOTHER verifies" — and the LAW OF RESOURCES: "better to burn tokens on a second angle than to ship something unverified and redo it."

Review to ideal (a builder layer)

Status. engine.review is a builder layer: supported by the schema and the exporter, but not yet part of the base flagship Quest seed (in development for the flagship). The prisms and verifyVotes above are part of base Quest; the polish loop described here is something you add yourself.

engine.review raises the bar above pass/fail: a loop of "improve → re-judge with prisms" until there is no finding at or above the severity threshold (minSeverity) OR the polish cap is reached (maxPolish, default 4). The recipe separates DONE (fit to ship) from IDEAL (nothing left to polish) — "don't confuse shipped with perfect".

Testing — provable tests (a builder layer)

Status. engine.testing (RED-FIRST + mutation gate) is a builder layer: supported by the schema and the exporter, but not yet part of the base flagship Quest seed (in development for the flagship). A purchased Quest does not bring this QA recipe out of the box; it's something you switch on when describing your own engine in the configurator.

engine.testing turns on a QA recipe with four hard rules:

• The test plan and cases are derived FROM the acceptance criteria (every required criterion → at least one case).
• RED-FIRST: the failing test first, then the code. "A test that has never failed has verified NOTHING."
• BANNED — tautology: a test MUST be capable of failing; expect(true).toBe(true) is forbidden.
• Mutation gate: run a real tool (StrykerJS for JS/TS, mutmut for Python). A surviving mutant means a hole in the tests. No tool available → FAIL-CLOSED-SKIP (honestly: the gate was skipped, not passed).
• Coverage is only a way to find holes, NOT a goal in itself.

Artifacts: TEST-PLAN.md and BUG-LOG.md.

A worked example

Input: the engine has written a SQL query for orders in a multi-tenant backend and says "done". In this example the engine was built with the engine.testing layer enabled (step 5) on top of base Quest (steps 1-4 and 6).

What the engine does:

1. Fan-out (Quest): implementation goes to one role; review goes to another (the skeptic).
2. Prisms, independently (Quest): tenancy looks at the query, finds SELECT * FROM orders WHERE id = $1 with no organization_id, and returns a verdict of block, critical ("cross-tenant reads are reachable"). security checks parameterization — ok. perf notes a missing index — minor.
3. Synthesis (Quest): a critical from the veto prism tenancy → an overall BLOCK, even though security and perf are nearly green.
4. loop-until-dry (Quest): the engine fixes the CLASS of problem — greps the whole module for queries without a tenant filter (not just this one), adds AND organization_id = $2 everywhere, and re-judges with tenancy only → now pass.
5. Tests (the engine.testing builder layer): derives the case "user B's query does not see user A's orders", writes it RED-FIRST (failing on the old code first), then green on the new one. The StrykerJS mutation gate kills the mutant that stripped out the tenant condition. This step is only available if you added the testing layer.
6. Second vote (Quest): the skeptical reviewer confirms — 2 independent "done" votes.

Result: the artifact ships only after the veto prism turned green; with the testing layer on, the regression is additionally locked down by a test capable of failing, and the mutant is dead. One check like this catches an entire class of cross-tenant leaks, not a single instance.

The honest caveat

First on status: prisms, verdict synthesis and the verifyVotes rule are part of base Quest. The engine.review polish layer and the engine.testing QA recipe (RED-FIRST + mutation gate) are builder layers, not yet included in the flagship seed (in development). The mutation gate from the worked example only runs if you added that layer.

Now on the limits of the guarantee. The mutation gate is ENFORCED only where a real tool actually runs (there's an exit code); otherwise it's an honest FAIL-CLOSED-SKIP. Prism synthesis and the vote rule are a deterministic rule, but their execution on Cursor/AGENTS without a runtime stays ADVISORY (the model follows the recipe; no exit code blocks anything). On Claude Code and in Cursor part of this can be pinned down with a hook; a git pre-commit hook and CI give an ENFORCED floor on any target. The configurator marks that boundary honestly.