AI now publishes proofs, records and benchmark results faster than anyone can read them. Verify re-checks a claim from its own files, borrowing nothing from the claimant, and answers in one word: certified, refuted, or refused. Every answer comes with proof you can run yourself.
0 of 322
planted false values certified
Before it is trusted, the decider is attacked: 572 pre-registered mutants of 119 published claims, each false one witnessed by a script that shares no code with it.
At a tolerance of 10⁻⁶, a floating-point check accepted 82 of the 295 witnessed-false values. The exact decider certified none of the false ones, and refuted none of the true ones.
34 of 242
formalized families claim more than they prove
A proof checker verifies the statement it is handed; nobody checks the statement against the claim. Read clause by clause, these prove a weaker or a different theorem than their headline, and nothing in the release says so, while the proofs themselves pass a second kernel.
45.6%
of MATH-500 with no edit at all, under the maker’s own settings
On 2 November MIT CSAIL posted that an AI had edited 0.016% of a small model’s weights and taken it from 7.9 to 48.1 on MATH-500, “up to 6×”. Three million people read it. The edit was decided from its bytes that night: two tensors differ, 130,188 entries, exactly as released. The untouched model was re-run under its maker’s own settings and graded exactly the next day, and the authors were told through their own repository the same afternoon.
The edit is real. The six times is mostly a settings effect. The audit →
“Carlos Toledo has independently verified the complete configurations in all 19 dimensions, confirming their point counts and exact dimensions.”the claimant, after an audit: the Qiushi Engine authors (Oxelra-AI)
Every verdict re-runs with one command, without this engine.
Each product is marked with where it stands: pilots open, live, or in development.
Reward and grading a policy cannot game: the ground truth is a certificate, in code you run, and the grader must refuse planted wrong answers before it grades anything.
89.55%
of 4,000 provably wrong answers pass a tolerance grader. A certificate grader admits 0. Where a grader ships twice, the two share no code and a disagreement stops the run; the second grader of the Weights audit caught a bug in the first on its first run.
An independent audit of a claim, before or after publication: the gaps listed first, every finite part decided exactly, every formal statement read against its headline.
34 of 242
formal statements in OpenAI’s math release claim more than they prove, while its 416 proofs pass a second kernel. The audit →
Send your reward function. We attack it with planted forgeries and report its false-accept rate per kind of forgery, with the fixes.
Our own decider, first: 572 planted forgeries, 0 certified; a float check at 10⁻⁶ let 82 through.
Answer keys and benchmark labels re-decided step by step; every wrong key comes back with the exact value that shows it.
GSM8K: 8,792 answer keys re-decided, 26 printed steps wrong by more than a rounding. The audit →
An RL environment whose reward is certified decimals: both ends of an interval proven, or nothing. Training a model on it waits on hosted access.
7 of 7 open models score below exact bisection: the best 0.099, bisection 0.340.
Send your rollout logs. Every rewarded sample is re-decided by an independent checker, and the reward paid to wrong answers is reported.
The re-checker is built on Verify Gym’s two deciders, which share no code: 2,600 answers compared, 0 disagreements.
Model answers that carry a certificate, so a fine-tune or an eval set holds only what was proven.
In development: a first set exists, each answer decided by both of Verify Gym’s deciders, and is not yet published.
Does a formal statement prove what its headline claims? The claim’s text goes in; a checkable object comes out, or a refusal with the reason.
The need: the 34 OpenAI statements above. The translator, claimbox, is open.
Prove you held exact bytes by a date without revealing them: a salted digest leaves your machine, the date comes from public anchors nobody here controls, and anyone re-derives it with one line of shell.
0 bytes
of the idea leave your machine, by design: the archive and the salt stay with you. The same sha-committed records already run the Forecast Gym. The battery passes; the tool is not yet published.
The product in four steps. An audit is the whole path for one published claim; a grader states a task family once and runs steps 3 and 4 on every answer a model gives.
89.55%
of 4,000 provably wrong answers admitted by a tolerance grader
A grader that compares decimals within a tolerance rewards answers that are provably wrong, and a policy trained against it learns exactly those. Tightening the tolerance by twelve orders of magnitude does not close the hole. A grader that reads the certificate admits none, and rejects nothing right.
Verify Gym and three more environments built on certificates run on Prime Intellect’s Environments Hub. The measurement →
Researchers now hold back directions for fear that a draft pasted into a frontier model comes back as someone else’s paper; Terence Tao said as much in his Math 2.0 talk. A seal lets you prove you held the exact bytes by a date and reveal nothing until you choose to. Seal before anything goes into a cloud model.
A deterministic archive of the file or folder and a 32-byte salt stay on your machine. One digest comes out: SHA-256 of the salt and the archive’s hash, nothing a reader can invert or guess.
The digest goes to OpenTimestamps (a Bitcoin-attested date) and to a public registry whose commit date is the second anchor. Neither anchor is ours to move.
Hand over the archive and the salt. Anyone re-derives the digest with one line of shell, without this engine, and reads the anchors’ dates.
CERTIFIED: the bytes re-derive the digest and an anchor confirms the date. REFUTED: they do not, and the input that differs is named. REFUSED: a part is missing or no anchor is confirmed yet, with what would decide it. An opened seal can go straight into Statement Check or an audit.
What a seal does not say. It proves possession of exact bytes by a date. It does not say who first had the idea, that the idea is correct, or anything about credit; a short statement can be guessed and hashed, so seal the whole draft, and a lost salt is an unopenable seal.
The value is in the ones that do not, and in saying exactly which part held.
Everything below runs on the same decider and the same three words, and most of it is open.
An asymptotic statement, a “for all n”, or a proof written in prose is refused with the reason, and the refusal says what would make it decidable.
Nothing a claimant wrote is run. The decision comes from code written apart from it, so a bug cannot certify itself.
A kernel cannot say what a statement means. Where a verdict rests on reading a statement, the report says so and keeps the reading apart from what was decided.
Absence of proof is never turned into CERTIFIED or REFUTED. Not peer-reviewed; every page ships its own falsifiers.
What it is not: a proof of inference. A zero-knowledge proof that a committed model produced an answer (Anthropic’s FlashProve, October 2026) says which model ran. Verify says whether the answer is right. One is the layer below the other; they compose, and neither replaces the other.
Send one published claim, or one task family. Before anything runs, you get the scope: what can be decided, what will be read, what will be refused, and why.
The first request through this page arrived on 3 November: an outside prover’s Lean bridge between two published refutations, replayed and certified the same day.
Every number on this page is read from a public record; the evidence is in the reports →