cert-machine · verified reward · every trajectory from the append-only ledger

The verifier in the loop

The reward channel, demonstrated in closed loop: a model proposes a decomposition, the grader answers every failure with its own refutation mechanism — the first violated equation and its exact discrepancy, nothing more — and the model retries in the same conversation. No hints, no partial answers, no human. This page renders every trajectory round by round; the question it measures is whether a certificate's failure mechanism is, by itself, a training signal.

tl;dr
  • The finding. 3 of 6 trajectories end CERTIFIED — every certification was first-shot, and no below-bar trajectory was rescued by feedback: the channel is honest in both directions, unable to coach and impossible to sweet-talk. The feedback is never advice: it is the exact index where the identity fails and the exact discrepancy, read off the certificate that refuted the attempt.
  • The mechanism. Reinforcement signal without reward hacking: the loop's reward is a certificate, so "the model satisfied the grader" and "the mathematics is correct" are the same event. The three feedback templates are fixed in the harness source, and this build REFUSES to render a trajectory whose feedback deviates from them — coaching would invalidate the demonstration.
  • Check it. python3 tools/llm-harness.py --model <id> --target "(3, 3, 3, 23)" --loop 6 --trajectories 1 --max-tokens 5000 --loop-ledger certs/matmul-loop-ledger.jsonl — same red and green controls as every campaign, run before any round.
§0 · the loop, drawn

Where the feedback did and did not convert

Each row is one conversation with the grader; each cell is one graded proposal. A row that turns green after a red cell is a conversion — the verifier's own mechanism, fed back, moving a model from wrong to proved. Look for one.

round 1 round 2 round 3 round 4 round 5 round 6 round 7 round 8 haiku-4-5 · (2, 2, 2, 7) #0 haiku-4-5 · (2, 2, 2, 7) #1 haiku-4-5 · (2, 2, 2, 7) #2 opus-5 · (2, 2, 2, 7) #0 opus-5 · (2, 2, 2, 7) #1 opus-5 · (2, 2, 2, 7) #2 rounds of the conversation, in order — one cell per graded proposal rejected malformed certified
6 trajectories, 27 graded rounds. 3 end certified and 0 of those were conversions — every certified row is green in its first cell. That is the honest state of this loop and it is recorded exactly as a success would be: the feedback is template-locked to the grader's mechanism, so there is no coaching to credit, and so far there is no conversion to credit either.
trajectories
6
claude-haiku-4-5-20251001 · claude-opus-5 — each an independent conversation against the grader
end certified
3
the terminal row of each is an exact theorem
converted by feedback
0
failed at round 1, certified by the round cap — the verifier's mechanism did the converting
rounds graded
27
every round a ledger row with its verdict and the exact feedback sent
coaching
0
feedback is template-locked to the grader's own mechanism; the build refuses on any deviation
§1 · the design

What the model is told when it fails

Three failure classes, three fixed messages. A reply that did not parse is told the required JSON shape and nothing else. A proposal that failed the prune-only screen is told which class of check it failed (shapes, rank bound, or float spot-test) and nothing else. A proposal the certifier REFUTED is told the first index where the defining identity fails and the exact value of the discrepancy — the certificate's own first_violation field, verbatim. That is the entire vocabulary of the feedback channel.

This is the verified-reward property in motion: the loop can only converge to a certificate, because certification is the only exit the grader rewards, and the grader cannot be satisfied by anything less than the exact identity. There is no rubric to overfit, no judge to persuade, no reference value to reproduce — reward hacking is not difficult here, it is undefined.

§2 · the trajectories

Round by round, from the ledger

claude-haiku-4-5-20251001 · (2, 2, 2, 7) · trajectory 0verdictwhat the verifier sent back
round 1rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 2rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 3rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 4rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 5malformedYour reply did not parse as the required JSON object {"u": [...], "v": [...], "w": [...]}. Reply with ONLY that JSON object — no prose, no markdown code fences.
round 6rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 7rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 8rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
claude-haiku-4-5-20251001 · (2, 2, 2, 7) · trajectory 1verdictwhat the verifier sent back
round 1rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 2rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 3rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 4rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 5rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 6rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 7rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 8rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
claude-haiku-4-5-20251001 · (2, 2, 2, 7) · trajectory 2verdictwhat the verifier sent back
round 1rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 2rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 3rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 4rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 5rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 6rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 7rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
round 8rejectedYour proposal failed the screen: wrong row lengths, more rows than the rank bound, or a floating-point spot-check mismatch. Recheck the stated convention and shapes, then reply with ONLY the corrected JSON object.
claude-opus-5 · (2, 2, 2, 7) · trajectory 0verdictwhat the verifier sent back
round 1CERTIFIEDnothing — the terminal row is a theorem
claude-opus-5 · (2, 2, 2, 7) · trajectory 1verdictwhat the verifier sent back
round 1CERTIFIEDnothing — the terminal row is a theorem
claude-opus-5 · (2, 2, 2, 7) · trajectory 2verdictwhat the verifier sent back
round 1CERTIFIEDnothing — the terminal row is a theorem

Every row above is read off certs/matmul-loop-ledger.jsonl at build time — the raw proposals and full certificates are in the ledger. A trajectory that ends without certifying is rendered exactly like one that succeeds; the demonstration is the channel, not a highlight reel.

§3 · why it matters

From eval to training signal

An evaluation grades one attempt; a reward channel shapes many. The loop is the smallest possible demonstration that this machine's grading is the second thing, not just the first: the same certificate that scores a proposal carries, in its refutation mechanism, enough signal to steer the next one — and because false positives are provably impossible, a policy trained against this channel cannot learn to exploit its grader. Scaled up, this is reinforcement learning on certified rewards for mathematical search; run once in a conversation, it is what this page shows.