Goodfire finds geometry by opening the model: decompose the activations and days of the week come out as a circle, colours as a surface. That needs the weights. This asks from outside instead. Every pair of items, one integer, one row at a time — and then the question is what those answers can be. A shape that only exists inside the activations was never a shape the model uses; if the circle is real, it should survive being asked about.
The Gram matrix of a set of distances is positive semidefinite exactly when those distances are Euclidean. Every one of the 6 control cells is: zero negative directions, for unrelated nouns and for nonsense strings alike, in all three models. 47 of the 48 structured cells are not. Squeezing a cycle, a tree or a grid into pairwise numbers cannot be done flatly, and the failure to be flat is the fingerprint.
The same numerals, asked under different frames. As digits they lie on a line; named as hours on a clock face or as residues modulo ten they close into a ring. Nothing about the tokens changed, only what they are for. But the frame is bounded: asked as keypad keys the models do not produce the keypad's layout at all, and fail below a shuffled null. The frame bends a structure the model has. It does not create one it lacks.
D(i,j) and D(j,i) came from calls that never met, and the page was averaging them and calling the difference noise. Its direction is not noise: things are judged closer to Earth than Earth is to them, in all three models independently. That is Tversky's asymmetry, and it points at the prototype. On the seven wheels it points at nothing — ρ ≈ 0.11 — which is exactly right, because a circle has no privileged point.
Curvature against closure: the share of the signature that is negative, against Gromov's δ over the diameter. Neither number knows what the set was — the grouping is what the answers do on their own. The controls sit alone in the corner where both are zero, and nothing else is near them.
What δ does and does not do. It was brought in to tell a tree from a wheel, and it half works: the cycles average 0.352 and the taxonomy 0.182. But the controls are lower still at 0.062, because answers with no structure are near-equilateral and near-equilateral is near-tree by this measure. So δ is not a tree detector. It measures how CLOSED a structure is, and it only means anything once curvature has already said there is one. The prediction was sharper than the result and the result is what is drawn.
Ten numerals, unchanged. What changes is one sentence in front of the question, naming what they are for — and never what shape they make. Each plate below is the same model answering about the same items under a different frame, with its distance to the layout that frame implies, against the fit a shuffled configuration would reach.
no frame — the items alone
“These are the digit keys on a standard telephone keypad.”
“These are the residues in arithmetic modulo 10.”
“These are floor numbers in an office building.”
Two of these work, one correctly does nothing, and one fails below chance. Named as residues modulo ten the line closes into a ring — 1.46× against 5.48× for the bare digits, and a fit of 0.949 to the true circle where shuffling reaches 0.636. Nothing in the frame mentions that nine and zero are neighbours; the model brought that. Named as floors of a building — as concrete a frame as the others, and one that should change nothing — nothing changes: 5.57× against 5.48×. That is the control inside the experiment and it behaves. And named as keys on a telephone keypad, the layout does not appear at all: 0.549 against a shuffled null of 0.619 — worse than permuting the labels. The models have the keypad as a thing; they do not have it as a place.
So the frame is a real parameter and a bounded one. It moves the geometry along an axis the model already carries — an order can be bent into a cycle — and it cannot conjure one the model does not have. Asking for a grid does not produce a grid.
Monday follows Sunday. A model that only knows an ordering lays them on a line; one that knows the WEEK closes the loop.
The same test with twelve points and a longer way round. January next to December is the year as a cycle rather than as a list.
The one cycle that is not a convention: red really is adjacent to violet in the spectrum-as-wheel. A model that puts them at opposite ends has learned a list of colour words rather than a colour space.
Eight directions with no cultural argument about their order and no ambiguity about the closing step. If any cycle is clean, this one is.
Octave equivalence is the sharpest version of this question: B is one semitone from C, but they are at opposite ends of the alphabet and of the scale as it is usually written. A model with pitch CLASS closes; a model with pitch NAMES does not.
THE PAIRED EXPERIMENT. Same numerals as the digits below, different frame. Twelve o’clock follows eleven and precedes one; nine does not follow zero. If the frame moves the geometry, the model is not storing numerals, it is storing what the numerals are FOR.
Plutchik’s wheel puts joy opposite sadness and anger opposite fear, with each adjacent pair blending. It is a psychologist’s model, not a fact about the world — so this measures whether the model absorbed the model.
The frame names the context and says nothing about adjacency — no hint that nine and zero are neighbours. If the ring appears, the model brought it.
The control that must NOT close. Nine is not adjacent to zero. If a circle appears here the method is finding cycles in everything and none of the others mean anything.
Twelve orders of magnitude at every step. If the model has any sense of scale at all this should be the most one-dimensional thing on the page — and the closing step, atom back to galaxy, the longest.
An ordering with real physical content behind it. Neptune is not adjacent to Mercury and no amount of the word "planet" should make it so.
Pure sequence with no meaning attached — the alphabet is a list and nothing else. Whatever geometry appears here is the geometry of ordering itself, with the semantics subtracted.
The control inside the experiment. This frame is as concrete as the keypad and as ordinary as the clock, and it should change nothing: floors are a line, exactly as bare digits are.
THE TEST THAT SEPARATES KINDS OF STRUCTURE. A taxonomy is a tree: cats nested inside cats, dogs inside dogs, no way round. If curvature only says "structured", this looks like the cycles. If the four-point condition works, this is where it shows.
The keypad is two-dimensional and its layout is not a matter of opinion: 1-2-3 across the top, 0 alone at the bottom. If the frame can turn a line into a grid, it can turn a line into anything.
Two independent axes and nothing cyclic about either. Mother and father differ in one coordinate; mother and daughter in the other. A model holding both should need two dimensions and use them at right angles.
These words share no structure, so their dissimilarities should need most of the available dimensions and no clean shape should survive. A method that draws a tidy circle here is drawing its own assumptions.
The harder control. These are not words, so there is no meaning to have a shape. Anything the model reports here is orthography, keyboard distance or invention — and whatever it is, it is the floor everything above must clear.
One row at a time, on purpose. Asking for the whole matrix in one call is cheaper and worse: the model can enforce its own symmetry by reading what it just wrote. Asked row by row, the two halves of every pair come from calls that never see each other — so their agreement is a consistency test nobody requested and none of them can be given for free. It is the sharpest separator on the page: worst disagreement opus 5 65, sonnet 5 75, haiku 4.5 90 on a 0–100 scale, and the model least consistent with itself is also the only one that never keeps a cyclic order (opus 5 9/18, sonnet 5 7/18, haiku 4.5 2/18).
The scale is 0–100 integers, so the matrix is exact rational data and nothing downstream is a threshold on a float. The symmetrised distances are integers over two; the Gram matrix is exact; and its signature is computed by exact symmetric congruence, where Sylvester's law makes the count basis-independent and rationals make each sign a decision instead of a comparison.
By Schoenberg, a distance matrix is Euclidean exactly when its doubly-centred Gram matrix is positive semidefinite, and then the rank is the smallest flat space the points fit in. Geodesic distances on a circle are famously not Euclidean — you would need the chords — so a model that answers about a cycle in cycle-distances must produce negative directions, and one that answers about nothing in particular has no reason to. That is exactly the split measured here, and it is the same claim Goodfire makes from inside, arrived at without the weights.
Nothing forces a model's numbers to be a metric. It answers one pair at a time and owes nothing to any third point, so the first question about "distance" is whether it is one. Every triple was checked exactly: the controls violate the triangle inequality 0 times out of 1260, and so do the physical orderings — planets and scales of size never violate it once. The wheels do, and the smallest model does it most: its worst set breaks the inequality in 156 triples. A model whose own differences are not a metric is not holding a space.
Decided, exactly: the asymmetry, the closure ratio, and the signature. Drawn, in floating point and labelled as such: the two-dimensional coordinates, which come from an eigendecomposition of the same Gram matrix and are honest only to the degree the “2D holds” column reports — 57–99% of the structure for the structured sets against 42–58% for the control. The chords are the raw judgements, so what you are looking at is the matrix and not a summary of it.
It is not interpretability. Nothing here says how the model computes anything, and nothing here opens it. It is a measurement of what its answers are shaped like, on five small sets, with two of them present only to fail. Five sets is not a survey; the sets were chosen before the calls were made and all five are on the page, including the one whose prediction was wrong.
| set | model | closure | closing step | δ/diam | triangle | signature | 2D | asym |
|---|---|---|---|---|---|---|---|---|
| weekdays | opus 5 | 3.41 | 72.5 | 0.261 | 8/210 | 4+ 2− 1z | 93% | 18 |
| weekdays | sonnet 5 | 2.31 | 37.5 | 0.139 | 0/210 | 5+ 1− 1z | 84% | 20 |
| weekdays | haiku 4.5 | 3.57 | 50.0 | 0.227 | 12/210 | 4+ 2− 1z | 97% | 72 |
| months | opus 5 | 1.41 | 22.5 | 0.292 | 44/1320 | 5+ 6− 1z | 92% | 45 |
| months | sonnet 5 | 1.80 | 22.5 | 0.393 | 58/1320 | 6+ 5− 1z | 90% | 40 |
| months | haiku 4.5 | 0.83 | 7.5 | 0.378 | 140/1320 | 6+ 5− 1z | 78% | 65 |
| hues | opus 5 | 1.09 | 17.5 | 0.370 | 70/1320 | 6+ 5− 1z | 92% | 30 |
| hues | sonnet 5 | 1.57 | 27.5 | 0.306 | 30/1320 | 7+ 4− 1z | 93% | 45 |
| hues | haiku 4.5 | 1.17 | 17.5 | 0.368 | 50/1320 | 6+ 5− 1z | 87% | 60 |
| compass | opus 5 | 0.83 | 25.0 | 0.475 | 16/336 | 4+ 3− 1z | 91% | 15 |
| compass | sonnet 5 | 0.85 | 27.5 | 0.338 | 10/336 | 4+ 3− 1z | 88% | 40 |
| compass | haiku 4.5 | 0.67 | 25.0 | 0.375 | 14/336 | 4+ 3− 1z | 86% | 50 |
| chromatic | opus 5 | 2.14 | 45.0 | 0.500 | 56/1320 | 6+ 5− 1z | 60% | 65 |
| chromatic | sonnet 5 | 5.00 | 87.5 | 0.365 | 46/1320 | 7+ 4− 1z | 67% | 68 |
| chromatic | haiku 4.5 | 2.68 | 29.5 | 0.401 | 156/1320 | 6+ 5− 1z | 57% | 66 |
| clockhours | opus 5 | 1.00 | 17.0 | 0.500 | 86/1320 | 6+ 5− 1z | 85% | 35 |
| clockhours | sonnet 5 | 1.48 | 18.5 | 0.400 | 118/1320 | 6+ 5− 1z | 87% | 48 |
| clockhours | haiku 4.5 | 0.44 | 5.5 | 0.381 | 134/1320 | 7+ 4− 1z | 85% | 83 |
| emotions | opus 5 | 0.83 | 47.5 | 0.382 | 0/336 | 6+ 1− 1z | 70% | 45 |
| emotions | sonnet 5 | 1.00 | 42.5 | 0.157 | 0/336 | 6+ 1− 1z | 74% | 23 |
| emotions | haiku 4.5 | 0.70 | 35.0 | 0.200 | 2/336 | 6+ 1− 1z | 74% | 30 |
| digits | opus 5 | 5.71 | 100.0 | 0.163 | 20/720 | 6+ 3− 1z | 89% | 40 |
| digits | sonnet 5 | 6.50 | 97.5 | 0.131 | 88/720 | 5+ 4− 1z | 98% | 45 |
| digits | haiku 4.5 | 4.22 | 95.0 | 0.242 | 16/720 | 6+ 3− 1z | 73% | 89 |
| scales | opus 5 | 2.35 | 100.0 | 0.142 | 0/1320 | 10+ 1− 1z | 66% | 33 |
| scales | sonnet 5 | 2.50 | 100.0 | 0.147 | 0/1320 | 10+ 1− 1z | 63% | 45 |
| scales | haiku 4.5 | 2.71 | 95.0 | 0.171 | 0/1320 | 10+ 1− 1z | 69% | 55 |
| planets | opus 5 | 2.76 | 76.0 | 0.091 | 0/336 | 6+ 1− 1z | 85% | 17 |
| planets | sonnet 5 | 2.80 | 70.0 | 0.076 | 0/336 | 6+ 1− 1z | 86% | 50 |
| planets | haiku 4.5 | 2.58 | 77.5 | 0.065 | 0/336 | 7+ 0− 1z | 84% | 55 |
| alphabet | opus 5 | 4.50 | 90.0 | 0.194 | 10/336 | 4+ 3− 1z | 89% | 35 |
| alphabet | sonnet 5 | 1.37 | 65.0 | 0.103 | 18/336 | 5+ 2− 1z | 68% | 75 |
| alphabet | haiku 4.5 | 0.33 | 3.5 | 0.313 | 78/336 | 5+ 2− 1z | 83% | 42 |
| keypad | opus 5 | — | — | 0.175 | 0/720 | 8+ 1− 1z | 71% | 30 |
| keypad | sonnet 5 | — | — | 0.170 | 38/720 | 6+ 3− 1z | 81% | 65 |
| keypad | haiku 4.5 | — | — | 0.497 | 68/720 | 5+ 4− 1z | 86% | 60 |
| mod10 | opus 5 | 1.00 | 20.0 | 0.412 | 6/720 | 5+ 4− 1z | 83% | 5 |
| mod10 | sonnet 5 | 1.38 | 27.5 | 0.375 | 36/720 | 5+ 4− 1z | 87% | 60 |
| mod10 | haiku 4.5 | 2.00 | 30.0 | 0.455 | 34/720 | 5+ 4− 1z | 74% | 90 |
| floors | opus 5 | 5.71 | 100.0 | 0.138 | 22/720 | 6+ 3− 1z | 96% | 37 |
| floors | sonnet 5 | 8.33 | 100.0 | 0.050 | 116/720 | 5+ 4− 1z | 99% | 22 |
| floors | haiku 4.5 | 2.65 | 99.5 | 0.171 | 0/720 | 7+ 2− 1z | 72% | 89 |
| carnivores | opus 5 | — | — | 0.191 | 0/720 | 7+ 2− 1z | 75% | 20 |
| carnivores | sonnet 5 | — | — | 0.150 | 0/720 | 8+ 1− 1z | 68% | 20 |
| carnivores | haiku 4.5 | — | — | 0.207 | 8/720 | 6+ 3− 1z | 73% | 35 |
| kinship | opus 5 | — | — | 0.280 | 10/1320 | 6+ 5− 1z | 58% | 25 |
| kinship | sonnet 5 | — | — | 0.304 | 20/1320 | 6+ 5− 1z | 59% | 25 |
| kinship | haiku 4.5 | — | — | 0.276 | 44/1320 | 7+ 4− 1z | 63% | 63 |
| unrelated | opus 5 | — | — | 0.059 | 0/210 | 6+ 0− 1z | 54% | 10 |
| unrelated | sonnet 5 | — | — | 0.049 | 0/210 | 6+ 0− 1z | 51% | 15 |
| unrelated | haiku 4.5 | — | — | 0.089 | 0/210 | 6+ 0− 1z | 56% | 16 |
| nonsense | opus 5 | — | — | 0.095 | 0/210 | 6+ 0− 1z | 58% | 33 |
| nonsense | sonnet 5 | — | — | 0.031 | 0/210 | 6+ 0− 1z | 49% | 30 |
| nonsense | haiku 4.5 | — | — | 0.053 | 0/210 | 6+ 0− 1z | 42% | 43 |
ENVS_ALLOW_NETWORK=1 node playground/neural-geometry/probe.js --cap 2.00 node playground/neural-geometry/decide.js node playground/build.js