The payload, and why it has a denominator
Almost everything written about compressing text for a model reports accuracy on real prose, where nobody knows how much information was in the source to begin with. There is no denominator, so how much got through has no answer.
So the payload here is manufactured. Each source is an incident report assembled from values drawn at random at the moment the run was made: a vessel from a list of sixteen, a cargo from eight, a draught from one of eighty-eight steps, and so on. That fixes its exact information content before anything is measured.
Your turn: you are the receiver
A sender saw one of these reports and a byte budget. You will see only what it wrote. You will never see the report. Then you answer ten questions about it, four options each, and you must answer all ten: where the message does not tell you, guess. That constraint is not a courtesy, and the reason is at the bottom of this page.
Then the bar chart shows every machine that read the same message you did.
Chance is 2.5 of 10, and chance scores exactly zero bits. Being right by guessing is not information arriving. A receiver that is confident and wrong has recovered less than nothing, which is why the score is bits and not a percentage.
What the machines did
Every condition, pooled across sources and across the receivers on the panel. The table is computed in your browser from the same recorded rows the verifier reads, so it cannot drift from them.
Bits recovered is a lower bound, by Fano's inequality: given how often a receiver was right on a four-way question, this much information must have reached it. It needs no assumption about the receiver's confidence, because it never asks. The maximum is 20.0 bits, being ten questions at two bits each.
The one clause that costs
Comparing a terse message against a plain English summary is not a measurement, because the two instructions differ in several ways at once. So the two arms here are a minimal pair: the same instruction, byte for byte, with one clause appended to the second.
Given the same report and the same budget, the free arm writes this:
and the legible arm writes this:
The price is not always paid in length. On other sources the legible arm came in shorter and paid in content instead, quietly dropping the weather, the heading, the crew count and the corroboration clause rather than spending the bytes to say them in full sentences. That is exactly why the unit has to be bits per byte and not byte count: the constraint can be paid for in either currency, and only the ratio sees both.
What the clause costs
Two ways to say it, both from the same rows.
At the same cost, less arrives. Comparing the two arms in the 128 to 255 byte band, where both have plenty of messages: the free arm delivers 17.92 bits [16.14, 18.91] and the legible arm 11.06 [9.56, 12.44]. The intervals do not overlap. That is a factor of 1.62 in information, bought with the same number of bytes. In the 64 to 127 band the gap runs the same way, 8.05 against 4.17.
At the same fidelity, more is spent. Writing short in one go, the free arm reaches 17.36 bits [14.58, 18.79] in 77 bytes. The nearest the legible arm comes is 16.21 bits [14.14, 17.61] in 205 bytes. Slightly less information, for 2.7 times the space.
And below 64 bytes it reverses, which was not expected. In the 32 to 63 band the legible arm nominally leads, 3.60 against 2.31, on intervals that overlap heavily, and below 32 bytes both are indistinguishable from zero. Nothing should be claimed from that except the shape of it: past a certain squeeze, notation stops helping. The free arm's abbreviating starts damaging what it abbreviates, and a short sentence that is still true beats a shorter one that has turned Dunlin into Dunnin and 8 souls into 8 lost. Both of those are real messages from this run.
Compressing twice is worse than compressing once
There are two ways to get a short message: ask for one, or take a longer message and ask for it to be halved, then halved again. The second is what a system does when it compacts a context it has already written, and it is much worse.
At around 78 bytes, a message written short in one go delivers 17.36 bits. A message squeezed down to the same size through successive halvings delivers 8.05. The legible arm shows the same thing at its own scale: 16.21 bits written short against 11.06 squeezed, at comparable length.
Each round of squeezing decides what to drop without knowing what the next round will need to drop, so the discards compound. The rate is the same and the loss is not.
One detail worth noticing in the free message above. The ship is called Dunlin. The free arm wrote Dunnin. Under pressure the abbreviating goes after the proper nouns, which are the highest-entropy items in the report and the ones a reader is most likely to be asked about.
The controls, and one of them did not do what it was supposed to
Two failures are already published for experiments shaped like this one, and both were built in before any number was believed.
The floor. A receiver shown no message at all, forced to answer anyway. It scores 0.289, on a 95 per cent interval of [0.205, 0.390] across 90 questions, which contains chance at 0.250. Any question it could reliably beat chance on would be answerable from prior knowledge rather than from transmission.
The mismatch. A real message, written by the same sender under the same instruction at the same length, but about a different ship, paired with this ship's questions. It collapses: 0.276, interval [0.233, 0.323] across 370 questions, again containing chance. So the score is not coming from the codec's habits, from the plausibility of the options, or from the mere presence of a confident-looking message. This is the control the floor cannot give, because the floor removes the message while this one keeps its exact shape and removes only its truth.
The scramble, which failed. The same message with its word order destroyed and every word kept. Bouchacourt and Baroni showed in 2018 that a sender and receiver can appear to be communicating while carrying nothing about what they were shown, and this is the control for that. It was supposed to collapse. It did not: across 390 questions the receivers scored 89 per cent on shuffled messages against 94 per cent on intact ones.
Read straight, that says something true and slightly deflating about what was found here. For this payload the message works as a bag of values. The syntax is not load-bearing, because each answer is a distinct lexical item and a receiver holding the right words in any order can place them. The models did not invent a grammar. They selected and abbreviated a vocabulary, which is a smaller claim than the one this page could have made, and it is the one the evidence supports.
Two things nobody was looking for
No model obeys a byte budget. Told a hard limit in bytes and told that messages over it are discarded, 2 of 112 complied. The median overrun was 2.71 times and the worst was 17.13 times, and the ratio got worse as the limit got tighter. The limit is not ignored, though: asked for 12 bytes a sender wrote 71, asked for 384 it wrote 359. A stated byte budget is a dial with a gain of about six, not a constraint. Everything here is therefore scored at the rate a message actually achieved rather than the one it was asked for, which is the correct way to build a rate curve in any case.
Models decline to guess, and decline most where it matters. Before the forced choice was added, 60 of 102 answer sheets came back as a refusal, and the refusals were concentrated exactly where the message carried least: the report is incomplete, I cannot answer. Scoring only the sheets that came back would have deleted the hardest trials from the weakest conditions and flattered every one of them. The polite version of the experiment would have produced a cleaner-looking result that was wrong.
What this does not show
One payload family, manufactured, whose facts are a list of independent values. Real prose is not that, and the vocabulary finding above may be a property of the payload rather than of the models. One sender. Free-tier models only, ten families, none of them frontier. Sample sizes in the hundreds of questions, not the thousands.
It does not show that a model invented a language. It shows that a model asked to be brief writes something denser than English, that the density is real because a blind receiver recovers the content from it, and that requiring a person to be able to read it costs something measurable.
The prior art matters here and is set out in full in the repository. Prompt and context compression is a populated field. What appears not to have been measured is the cost of the human-readability constraint as a quantity, which is the only thing this page claims as its own.
Apparatus, every recorded message, every answer sheet, and the verifier: research/price-of-legibility/. The run is append-only and resumable, and the report script re-derives every figure offline from the committed rows, so none of it needs to be taken on trust.