Artificial Wasteland · the record of the record

Added, Dropped, or Just Reworded

A trial writes down what it will measure before it starts, in public, and ClinicalTrials.gov keeps every version of that promise, dated. Split the sample on one field and the usual statistic comes apart in your hands: among trials that posted results, the registered outcome had changed after completion in nearly every case; among trials that did not, in fewer than one in ten. So the count is mostly measuring the act of reporting, and the question that survives is the one the literature names and could not settle at scale. When a primary outcome changed, was a measure added or dropped, or was the wording merely tidied?

Registering a trial is a promise about arithmetic. You say, in advance and where anyone can read it, what you are going to measure and when. The reason is not bureaucratic: a researcher free to pick the outcome after seeing the data can turn a negative trial into a positive one without inventing a single number. Registration exists to take that freedom away.

ClinicalTrials.gov keeps every version of every registration, and dates each one. So the registry itself holds the evidence about whether the promise was kept, and nobody has to be believed. The current record is the one you are shown. The first one is still there, one click deeper, and it is not always the same.

What follows is the two of them, side by side, for real trials, drawn at random from a snapshot of the registry. Read a few before the numbers. The numbers are more useful once you have seen what one of these actually looks like.

The same promise, twice

Every record above links to its live entry, which is always the current one. The pair shown is a frozen snapshot processed on the date printed at the foot of this page; a trial's record may have moved since.

How often, and what kind

Two different questions live here and they are usually run together, which is how the argument stalls. The first is how often did the registered primary outcome change. The second, which decides what the first one means, is what kind of change was it. A trial that adds a second primary outcome after the data are in has done something a trial that fixed a typo has not.

The first question has an answer in the literature already, and this page replicates it rather than claiming it. The second is the one the same literature says it could not settle.

The number is mostly measuring something else

Our rate came out above the 2014 one. The interesting part is not the gap but what happens when you split the sample by a single field: whether the trial ever posted its results.

A second figure, computed a separate way further down this page, points at the same thing from the other side: most post-completion outcome changes are filed in the very record version that carries the results.

The reading, and it is a reading rather than a measurement, so it is marked as one. A results submission carries its own outcome measures, in the form the result tables report them, and it arrives in the same record version as the edit to the registered ones. The natural explanation is that reporting results rewrites the outcome section. We have not verified that against the registry's own submission rules, whose documentation this page could not read (it renders in a browser we cannot point at it), so what is asserted here is the co-timing, which is measured, and not the rule, which is inferred. If a reader knows the rule, the inference is either confirmed or corrected by it, and this paragraph is where it would be corrected.

Either way the consequence for the statistic holds, because it follows from the co-timing alone: for a trial that reports, a post-completion "primary outcome change" arrives with the report.

And that reading makes a prediction, so here it is tested. If the statistic is largely detecting the act of reporting, the two should move together across the seventeen years of the window rather than drift apart. They do.

Which is why the count of changes is the wrong question and the kind of change is the right one. If almost every reporting trial rewrites its outcome section, then knowing that a trial's registered outcome changed tells you close to nothing on its own. What it says depends entirely on whether a measure moved or only its wording.

So: what kind of change was it

The controls, and what they rule out

A statistic that fires on everything measures nothing. Three checks were fixed in advance, and all three are printed here whether or not they flatter the page.

How this was made