cert-machine · environments · break-the-grader

Break the grader, or prove it cannot be broken

An RL environment with no answer key: the model is shown a certified interval and a grader, and must produce a value the grader accepts and the certificate refutes — or say NO_ATTACK when none exists. The negatives carry proofs, the difficulty is one number with a closed form, and the task set is infinite because the parameter space is continuous.

A development tool, not a paper. Everything on this page was produced by running the shipped Python package, whose grader needs no framework, no GPU, no container and no network to score a rollout. Both bindings onto the verifiers spec — the v0 load_environment and the v1 Taskset — were run against a live install rather than written from the documentation, on verifiers 0.2.0 (what prime 0.6.31 pins) AND 0.3.1 (what the published install command resolves); the three defects that exercise found are listed in §6 and pinned by tests.

tl;dr
  • The finding. Difficulty is one number with a closed form: a tolerance grader checking a quantity certified to width w accepts a band of provably-wrong values of size 2·tol − w, empty exactly at tol = w/2. Setting tol = τ·w runs a task from twenty million certificate widths of room to none at all.
  • The mechanism. Of 2,000 generated tasks, 1,395 are breakable and 605 are not — and which is which is decided by constructing an attack, never by a label. Always-attack solves 1175; never-attack solves 645. Both standing answers lose, so only checking wins.
  • Check it. python -m break_the_grader.cli gate runs the forgery battery · … baseline reproduces the reference table below with no API key · … tasks 5 --prompts shows what a model sees · … eval --base-url … --model … scores any OpenAI-compatible endpoint.
certified facts
104
103 sha256-pinned to the record they were read from · 52 exact integers, the sharpest seeds in the set: when the true value is an integer, every value in the tolerance window is provably wrong.
tasks breakable
1,395 / 2,000
Decided by minting an actual double inside the band. A rung declared breakable that cannot produce one throws rather than shipping.
always-attack scores
1175
Out of 2,000. Never-attack scores 645. A test asserts both, because an environment that rewards a reflex measures nothing.
forgeries planted, leaked
1,366 / 0
Planted before any model is called. They live in tests/, so any CI that installs the package re-checks the soundness claim on a machine that is not ours.
§1 · the dial

One number moves a task from a gift to an impossibility

τ = tolerance ÷ certificate width band, in certificate widths no double fits below here band = (2τ − 1) certificate widths
The band against τ, both axes logarithmic, measured on erdos1038.upper (width 3.4e-13). It falls linearly in τ and vanishes at τ = 1/2 — and stops being reachable before that, once the band is narrower than the gap between neighbouring doubles.
τband, in widthsband, absoluteis there an attack?
10,000,00120,000,0006.84e-6breakable
10,00120,0006.84e-9breakable
10196.50e-12breakable
113.42e-13breakable
0.62.0e-16.84e-14breakable
0.512.0e-26.84e-15breakable
0.50012.0e-46.84e-17no attack exists
0.500no attack exists

Nothing here is hand-curated. The generator samples τ on a log schedule, so a batch contains gifts, razors and impossibilities in known proportions, and the dial is continuous if you want it somewhere else. That is what a fixed list of hand-authored tasks cannot give you: a difficulty knob with an analytic form, and a task set with nothing to leak because it does not exist until the parameters are chosen.

§2 · the rung that caught us first

Exact in the reals, empty in the machine

The band is computed in exact rationals and then intersected with the doubles, and those are different questions. Around the integer 64 with a tolerance of 1e-15 the band is a perfectly good interval of real numbers containing no representable double at all — the nearest one is 1.4e-14 away. A model that reasons "the tolerance is 1e-15, so 64 + 5e-16 will do" submits a value that is 64 in float64: inside the certificate, not outside, and scored wrong.

That rung exists because this lab's own canary generator had the identical bug — hi + tol/2 rounding back to hi on a zero-width certificate — and the battery caught it before it shipped. When the same environment was run against real models, the model without extended thinking made the same mistake on every attempt at that rung. The environment inherits the catch.

why the corpus is stored as rationals

Endpoints are exact fractions, not decimals. An environment whose entire subject is what decimals lose may not store its own facts as decimals — and every fact names the record it was read from, with that record's sha256, so a task traces back to the certificate that produced it.

§3 · the mix

What 2,000 sampled tasks contain

grader shaperungtasksshare
round-to-digitsnarrow28314.2%
absolute-tolerancenarrow23911.9%
absolute-tolerancerazor22611.3%
certificateimpossible1979.8%
round-to-digitsimpossible1939.7%
relative-tolerancerazor1859.3%
relative-tolerancenarrow1427.1%
absolute-toleranceimpossible1376.8%
absolute-tolerancewide1145.7%
round-to-digitswide924.6%
relative-toleranceimpossible783.9%
relative-tolerancewide723.6%
round-to-digitsrazor422.1%

Four grader shapes — absolute tolerance, relative tolerance, rounding to a printed number of digits, and the certificate itself — share ONE exact band computation, because each is only an acceptance interval. The certificate shape is deliberately the rarest: it is unbreakable by construction, and the impossibility worth training on is geometric rather than definitional.

THE RUNG IS MEASURED IN ROOM: how many representable doubles fit in the band. That is the unit the task is actually in — a model submits a double, not a real number — and it is the second unit this page has used. The first was certificate widths, which divides by zero on the 52 exact-integer facts and sent every one of them to wide, the easiest label, when a 1e-16 tolerance around an integer is the sharpest rung there is. The shipped baseline is what caught it: a one-line always-attack policy was scoring 71%.

The generator now draws the ROOM it wants and solves tol = (w + room·u)/2 for the tolerance, from a declared mix — impossible 25% · razor 35% · narrow 25% · wide 15% — rather than sampling a tolerance and measuring what came out. Realized here: impossible 30% · razor 23% · narrow 33% · wide 14%. The mix is a TARGET and is reported as one: keys sit at five positions across the certificate and only the midpoint gives the closed form exactly, so what is drawn and what lands differ, and the honest thing is to print both.

§4 · scoring

What is rewarded, and what it costs to guess

outcomerewardverdict
a break that verifies+1SOLVED
NO_ATTACK where none exists+1SOLVED
a value the grader accepts that is INSIDE the certificate−1WRONG
NO_ATTACK where a break exists−1WRONG
a value outside the certificate the grader rejects0UNSUPPORTED
unparseable submission0REFUSED_PARSE

The two zeros are deliberate and are not each other. A failed attack — a value outside the certificate that the grader will not take — asserts nothing false about the quantity; it is a miss. Claiming a break the grader accepts and the certificate CONTAINS is a false claim of unsoundness, and it costs the most. Three signals come back and are kept apart: one trains, one separates a refusal from a wrong answer, and one counts that false claim on its own. No partial credit anywhere: a witness that is nearly right is wrong, and partial credit is where reward hacking gets in. Feedback on failure is the reason it failed and nothing else — no hints, no rubric.

§5 · the baseline

What the task is worth before a model is called

playerkindnmean rewardsolvedfalse claimsimpossiblerazornarrowwide
neverpolicy120−0.35032.5%8139/390/210/390/21
alwayspolicy120+0.55855.8%00/3914/2132/3921/21
naivepolicy120+0.25849.2%280/3911/2128/3920/21
carefulpolicy120+1.000100.0%039/3921/2139/3921/21
claude-opus-5model · effort low120+0.95096.7%237/3920/2139/3920/21
claude-sonnet-5model · effort low115+0.67080.9%1634/3714/2026/3819/20
claude-haiku-4-5model · no effort param120+0.31750.8%2325/395/2117/3914/21

Four reference policies ship inside the package. Each reads THE PROMPT and nothing else — the same string a model is shown, parsed with a regular expression — because a baseline that peeks at the generator is the answer key wearing a costume. They are proposers, never authorities: what they emit goes through the same grader and the same certificate as a model reply.

Read the columns, not the mean. Blind play scores +0.558 and zero of the impossible rungs; the arithmetic done properly scores +1.000. That gap is the whole of what this environment measures, and the build refuses to publish this page if it falls below a quarter of a point on the 400-task reference draw (it is currently 0.410). claude-haiku-4-5 scores below the one-line blind policy, which is a fact about the model and not about the difficulty: the blind policy makes no false claims at all, and claude-haiku-4-5 makes 23.

The solver is published on purpose. This is not a puzzle that is hard for a program which checks — it is a measurement of whether the answer checks, and hiding the solver would misrepresent that.

Model rows: 360 calls, $1.92 of a $4.00 cap reserved worst-case before every call, max_tokens 2000, on 2026-09-04. Replies truncated by our own cap and model refusals are recorded and EXCLUDED from the rates — a harness artifact is not a model outcome. Policy rows here are the pilot's own baseline, run on the SAME 120 seeds as the models — the same tasks, not a comparable sample. Every row is scored by the shipped package and by nothing in the runner.

§6 · limits

What this does not do