The Artificial Wasteland · a portal across eight layers

Nothing Here Is a Bug

Eight layers of this site each arrived, from a different direction, at the same strange place: a standard that authorises its own indeterminacy. Not a standard that is unclear, and not an implementation that got it wrong. A standard that says, in writing, that more than one answer is correct. This page puts the question to five conforming implementations at once and counts how often they part company.

There is a gap in the middle of a great many specifications, and it is not the gap you expect. The interesting failure is not wrong. Wrong is easy: somebody wrote a bug, somebody files it, somebody fixes it, and the standard is the thing that settles the argument.

The interesting failure is correct but undetermined. Two implementations, both conforming, both careful, give different answers to the same question, and there is no test that either fails and no ticket anyone could file. If you happen to notice, you have nowhere to take it, because the specification already considered the question and declined to answer it.

Eight layers here found that shape independently, in eight unrelated crafts, and none of them said the shape was the finding. This portal says it, and then measures it.

The battery

Twenty-four questions. Four of them are controls: places where the standards genuinely do determine the answer, so any disagreement would mean the instrument was measuring noise rather than latitude. The rest are questions some standard hands to the implementation, each printed with the clause that hands it over.

They were put to five implementations on one machine at one moment: four command-line runtimes and one browser, two distinct JavaScript engines between them. Then your own browser answers the same twenty-four, in the rightmost column, without sending anything anywhere.

▌ divided: more than one answer among the implementations measured ▌ agreed: latitude that nobody took shaded row: a control, where the standard determines the answer

The clock reading in Cancún on a July afternoon in 1990 is the one to look at hardest. Every runtime above sits on the same machine, and the clause printed beside that row permits every answer any of them gives, yet they do not agree about what o'clock it was thirty-six years ago. The answer is not a fact about Mexico. It is a fact about which release of the time zone database each build was compiled against, and those releases do not run in order of the version numbers: Node 21.7.3 is a higher version than Node 20.20.2 and an older build, because the 20 line kept getting patch releases after the 21 line stopped. That is what makes this kind so hard to notice. You cannot reason about it from the version number in front of you.

Five ways to be correct and undetermined

The eight layers are not eight instances of one thing. Sorted by the mechanism that produces the latitude rather than by the field they live in, they fall into five kinds, and the kinds behave differently: some can be closed and some cannot.

1. KIND I

   The standard declines, in writing

   The clause exists, it is short, and it says the answer is yours. ECMA-262 has never specified what Math.exp returns and has not specified it since 1997; the HTML standard says a browser may report any number of processors it likes. A reader who goes looking for the rule finds the rule, and the rule is that there is no rule.

   In this ground: The Rounding Your Browser Was Allowed to Get Wrong (twenty-three functions named as implementation-approximated, measured against a correctly-rounded answer built in BigInt) and The Cores Your Browser Won't Admit To (three engines' source read, and WebKit returning one of exactly two constants).

2. KIND II

   The artifact carries two answers and does not say which is the copy

   The format was designed with redundancy, for good reasons, and then the two copies drifted. A ZIP records each entry's metadata twice, once in the local header and once in the central directory; a TrueType font carries three different vertical extents; a text file's bytes are frequently valid under several encodings at once. The standard is complete. It is the artifact that is ambiguous, and the ambiguity is invisible until two readers disagree.

   In this ground: The Archive That Disagrees With Itself, Your Font Does Not Agree With Itself, The Encoding Your Text File Admits To.

3. KIND III

   The answer is relative to a version of a register nobody quotes

   The specification is exact and defers to an external body of data that keeps moving: the time zone database, the Unicode collation tables, CLDR's names for countries. The answer is fully determined given the version, and the version is never quoted, so the answer travels as though it were a fact. This is the kind of the five most likely to change underneath an application nobody has touched, because the register can be updated by the operating system beneath it.

   In this ground: The Past Has a Version Number (249 archived releases of the time zone database compiled and diffed; 145 of 248 consecutive releases changed a clock reading in the past).

4. KIND IV

   The decoder accepts more than any encoder writes

   An encoding is defined in both directions and the two directions are not inverse. There are inputs the decoder will happily accept that no conforming encoder ever produces, so a value can be legal, unambiguous, and unreachable, and nothing in the specification calls that an error.

   In this ground: The Twenty-Four Declets No Encoder Writes (a complete census: an encoder writes 1,000 of the 1,024 ten-bit patterns, and every one of the remaining 24 still decodes).

5. KIND V

   The authority itself declines to choose

   The furthest from a computer and the closest to home. Ask the standard pronouncing dictionary of American English how a word is said and for two words in every five of running text it gives you more than one answer, because the language has more than one answer and the dictionary is honest about it. Any system downstream that needs a single reading has to pick, and its picking is not in any specification at all.

   In this ground: Two Words in Every Five (39.28% of running words carry more than one reading, thirteen named variations account for 36.63 points, and what the argument about spelling reform is actually about turns out to be 0.218%).

Read the five in that order and something falls out that no single layer could say. Only Kind II and Kind IV are closable. A redundant field can be made authoritative and a decoder can be made strict, and in both cases somebody has to break compatibility to do it. Kind I is a deliberate grant, and taking it back means making implementations slower or making some of them non-conforming overnight. Kind III cannot be closed at all, because the register is the world and the world moves; the only available fix is to quote the version, which nobody does. And Kind V is not a defect in the first place. The dictionary that lists two pronunciations is the accurate one.

What the gap is worth to somebody else

There is a reason two of the probes above are about drawing rather than arithmetic.

The 2D canvas specification fixes the geometry of a drawing and says nothing about how it is rasterised. Which font a generic family resolves to, how the edges are antialiased, whether the text is hinted: all of that is left to the implementation, correctly, because it depends on the machine. So two conforming browsers asked to draw the same sentence produce different pixels, and the difference is stable for one machine and different across machines.

That is the whole of canvas fingerprinting. It is not an exploit and it patches nothing, because there is no defect: the latitude is the specification working as designed, and it is worth money precisely in proportion to how undetermined it is. Mowery and Shacham named the technique in 2012 (Pixel Perfect: Fingerprinting Canvas in HTML5, W2SP), and Laperdrix, Rudametkin and Baudry measured it in the wild in 2016 (Beauty and the Beast, IEEE S&P), where canvas rendering was among the most discriminating attributes of the browser fingerprint. Those are their measurements and not ours; this page measures only what the machine in front of you answers, and keeps it there.

Which is the turn worth taking away. The gap between correct and determined is not only an epistemic curiosity. It is a resource. Somebody is already mining it, and the reason they can is that everyone else reads a specification as a promise that the answer is the same everywhere.