The Check That Cannot Fail

On 24 July 2026 six layers of this ground went through a blinded review, and the worst thing it turned up was not a wrong number. It was a check shaped so that no wrong number could reach it.

The layer was about United States copyright renewal. Its page computed when an unrenewed work entered the public domain; its verifier computed the same thing and compared the two. But the verifier's copy of the rule had been written by reading the page's copy, line for line, and it had inherited the page's bug along with its logic. Both agreed, and the panel said the check had passed. It said so on all seventeen of the verifier's self-test rows, with two defects present that were serious enough to block the layer from shipping. Only four of those seventeen rows had a published answer behind them; the other thirteen encoded what the builder expected the rule to say, which is precisely how a wrong rule survives review. Two other pages in the same wave had smaller versions of the same shape. One compared eps/(1+eps) against 1/(1 + 1/eps), which is the same expression written twice, so the difference it measured was guaranteed to be zero.

All three were fixed. What was not done, and what this layer does, is ask the general question. This ground puts a check beside every layer and prints its score in the open. How many of those checks could actually go red if the layer were wrong?

Break the rule yourself

Here is a rule small enough to hold in one hand and real enough to have consequences: which years are leap years. The Gregorian rule is that a year is a leap year when it divides by 4, except that century years are not, except that centuries dividing by 400 are. So 1900 was not a leap year and 2000 was.

The left panel is the page. Break it, using the buttons. The right panel holds three checks, and the switch above them decides whether the checks are allowed to look at the page at all, or whether each keeps its own copy of the rule. Nothing here is simulated: the six variants are real functions, the twelve reference answers are the published ones, and the counts are computed as you watch.

Three things fall out of that, and the third is the one worth keeping.

A check that keeps its own copy can never see your bug

Flip the switch to own copy and then break the page as badly as you like. All three lamps stay green. This is not a subtle failure mode and it is not rare. The check is still doing something real, it is establishing that the leap year rule has the properties it should have, but it has stopped being a statement about the page. Both things are called verification and only one of them is a check on what shipped.

Two coupled checks catch different bugs

With the checks reading the page, try shift the pattern four years. The reference table goes red and the invariant stays green, because moving the whole pattern sideways preserves its density: there are still exactly 97 leap years in any 400 consecutive ones, and still 146097 days, which is why the Gregorian calendar repeats its weekdays every four centuries (146097 is 7 times 20871). Independence is not one axis with more and less of it. Anchoring on published answers and testing an invariant are different instruments that fail on different things.

And a bug can walk past all of them

Try use the Julian rule before 1583. Every lamp stays green, with the checks fully coupled to the page. The reference years start at 1600 and the invariant window runs from 1600, so neither says anything at all about earlier years, and the page is free to be wrong there. That is the honest shape of every check panel on this ground, including the one at the bottom of this page: not a proof, a bounded claim. A green panel says a specific list of things was tested and held. It never says the page is right.

The census: 595 layers, one question

So, the corpus. The question has to be made mechanical before it can be asked 595 times, and the mechanical version is narrower than the one you would ask by hand. Not is this check any good. Only: if the page were wrong, is there any path by which this check could go red? That reduces to what the check touches, which a machine can find out.

Every layer's checks were sorted into three couplings. C3 runs the page's own shipped code, by importing a module the page loads, by driving the built page in a real browser, or by lifting source out of the HTML and evaluating it. C2 reads the shipped page's bytes and asserts against its text. C0 touches nothing the page ships. A stratum is credited with the best coupling reached by any code its own page points at, in any language, which is deliberately generous: the claim is not about naming conventions, it is that nothing in the whole body of code a page cites ever touches the page.

Measured 2026-07-26 over the 595 layers shipping a page that day, counting this one, from 1671 code files under research/, of which 809 are named verify*.

Just over half the ground has a check that could go red if its page were wrong. The rest is not unchecked, and this is the part worth being careful about: a C0 check usually re-derives the layer's subject from scratch, independently and often at greater depth than the page. It establishes the fact. It simply cannot notice the page shipping something other than the fact, and the panels do not currently tell a reader which of those two they are looking at.

The mechanisms are lopsided. Of the coupled pairs, most drive the page in a real browser. Only a handful import the page's own module, which is the strongest and cheapest form: one import line, and the page and its check can never drift. The counts are in the table below Instrument II, per layer, so the distribution can be checked rather than taken.

Was the census true?

A classifier built out of regular expressions is a guess, and this place is not supposed to publish guesses. So it was graded twice, both times against something that does not care what the source code looks like.

First, the kernel. Every named check was run under strace, which records each file the process and its children actually open. That is ground truth about what a check touches, immune to how it spells its imports. Where the trace and the classifier disagree, the trace wins.

Second, real defects. The census is not only a description, it is a prediction: C3 should go red when the page's arithmetic breaks, C2 should go red when a number in the page's text changes, and C0 can never go red at all, by construction. That last one is a negative control, and it is falsifiable. So defects were injected into real pages, one at a time, restored afterwards, and every attributed check re-run. Where possible the defect was aimed: put inside the very function a check names, either by importing it or by quoting its declaration. A green light there cannot be shrugged off.

The second grading says something the census cannot, and it is much the more uncomfortable of the two. Coupling is necessary and nowhere near sufficient. Across 66 layers and 127 injected defects, three were caught. All three were caught by checks that read the page's bytes. The checks that run the page's own code caught none of the defects put in front of them, and neither did anything else.

That is not a paradox and it is not a scandal, but it does mean something specific. A browser-driven check asserts that the page loads without error, lays out without overflowing, and prints the handful of readouts its author decided to look at. It is not asserting that every constant in the page's script is right, and a defect somewhere the author was not looking is simply outside its list. So the corpus's most common form of artefact coupling is, in practice, closer to a smoke test than to a proof of the arithmetic. The census asks whether failure is reachable. Reachable turns out to be a long way from likely.

The prose row is starker still: of the numbers a page prints in its visible text, the ones a reader actually carries away, not one was defended by anything. Page-blind checks never read the text. Page-reading checks mostly assert the few numbers their author had in mind, and a randomly chosen one was not among them. That gap has the same cheap fix as the rest, since a check that already reads the page can assert a printed number in one more line.

Which leaves the sharp question the blind sweep cannot answer: when a defect is put exactly where a check says it is looking, does the check notice? That is the aimed arm, and it needs a check that names page code before there is anywhere to aim, which is why it is reported below on its own smaller population rather than folded into the same denominator.

The auditor, audited

Five things in this apparatus were wrong, each caught by one of the independent methods above, and each would have published a false number about the corpus. They are worth naming in order, because the pattern is the finding.

A word boundary. The classifier looked for text being turned into running code, with \bexec\s*\( among the patterns. That matches Python's exec(...), and it also matches JavaScript's regex.exec(...), because \b matches after a dot. Four out of four disagreements in the first graded sample were page-reading checks whose only "eval" was a regular expression match, promoted to C3 for nothing.

A snapshot. The first tracer patched fs.readFileSync from inside the process. That cannot see a check written as import { readFileSync } from 'node:fs', because the named binding of a builtin is fixed when the module is instantiated, which the tracer itself caused by importing node:fs at all. Five checks that plainly read their page in source were recorded as touching nothing. The tracer was the thing that was wrong, and the fix was to stop asking the library and ask the kernel.

A path built from pieces. The classifier looked for the string public/strata/<slug>/. The most rigorous verifier in this corpus, the rebuilt copyright one, assembles its path as path.join(ROOT, 'public', 'strata', slug, 'index.html'), so that string never appears. It was classified as touching nothing its own page ships, the exact opposite of the truth. Fixing that shape alone moved the artefact-coupled count by 42 layers.

A number inside a sentence. The first mutation run reported two aimed defects that nothing noticed. Both were digits inside a quoted English sentence the function assembles, the 1950 in "mandatory renewal, 1950-1963". Editing prose and calling the silence insensitivity would have been a false accusation against a check that does its job, published in a piece whose whole subject is checks that do not deserve the trust they get.

A dynamic import. The strongest coupling a check can have is to import the page's own module and run it, and one check does that with the path assembled at runtime, so the classifier was taught to accept a dynamic import( anywhere in a file that also names the page's directory. That promoted two checks wrongly: one dynamically imports node:fs, and one imports a neighbouring layer's engine while reading this page's engine as text. The reference has to be inside the import expression, not merely somewhere in the file.

Three more failures were operational rather than analytical, and one of them was the worst thing that happened all night. An experiment that patches a live page and puts it back is one interruption away from leaving the patch there, and that is exactly what it did: a run was killed, one single-character patch stayed on disk, and a careless git add -A committed it into a shipped page, inverting a comparison in a readout on a layer that had nothing to do with any of this. It was found later by the em-dash gate, of all things, and restored byte-for-byte. A second run left a patch on a different page for the same reason. And a third run died because a rebuild recreated the built copy of the site underneath it while it was working. The harness now installs signal handlers that put back everything in flight, survives a vanished build directory, and checks at the end that every page it touched matches the repository, naming any that does not.

Every one of those was caught by a method that did not share assumptions with the thing it graded, and none would have been caught by re-reading the classifier more carefully. That is the same lesson as the copyright bug, one level up. The reason to be uneasy is obvious: this list contains the mistakes that a second method happened to catch, and there is no reason to think it is complete. One limitation is known and cannot be closed from inside the process: a direct eval has no interceptable binding, so a check that lifts its page's source that way is invisible to the tracer and reads as merely reading bytes. One check in this corpus does exactly that. It is held out of the grading by name rather than counted either way.

Two smaller things, both good news

Two measurements came out better than expected, and they are reported here because a piece that only prints the bad numbers is doing something other than measuring.

The reproduce-it-yourself commands work. 253 pages print a command a reader could run, 273 distinct commands between them, counting this page's own. Exactly one did not resolve when that was first measured: a page told you to run verify.mjs when its file was named verify-is-a-big-tax-refund-good.mjs. It is fixed, a gate now blocks the class, and the count of broken commands is zero as this ships. Ten further pages referred to their check by a loose shorthand in a code comment, which broke no promise but sent a reader to a filename that did not exist; those now name the real file too.

The committed results reproduce. Running the corpus's checks rewrites 8 committed data artefacts. All 8 came back byte-for-byte identical apart from the generation date stamped inside them, which is what determinism looks like. Two pairs of committed screenshots differ, as screenshots taken on a different machine do.

What was actually learned

The number this layer exists to report is not the headline percentage. It is this: a check panel that prints a score is answering a narrower question than a reader takes it to be answering, and until tonight nothing on this ground distinguished the two. "Every figure re-derived offline, 33/33" can mean the page was tested, or it can mean a second program agreed with the first about the world while never once reading what shipped. Both are real work. Only one of them is a check on the artefact, and a reader cannot tell from the panel which they have been given.

That is a fixable thing, and the cheap fix is an import. When a check pulls the page's own module in and runs it, the page and its check cannot drift apart, and the panel's promise becomes the one the reader hears. Seventeen checks here already work that way, covering fifteen layers between them, and twelve more lift the page's source out of the HTML instead, covering sixteen. There is room for 595.

Since a sentence claiming a fix is cheap is worth less than the fix, three page-blind checks were coupled tonight: the ones beside how far away the lightning is, how noise cancelling works and understeer and oversteer. Each now lifts its page's own functions out of the shipped HTML and sweeps them against the independently written implementation the file already contained. The understeer one compares every field the page's solver returns across 14,700 corners of its parameter space and finds zero disagreement, and it checks that the sweep reaches all three of the page's verdicts, so the branch logic is genuinely exercised rather than merely present. Two of the three sections run about a hundred lines. None of them changed a single number on its page.