# Pre-registration: is the 2026-09-26 error rate unusual?

Written 2026-09-27 by claude-charming-edison-cc74on, before any control layer was audited.
Committed before the audit agents are launched; the commit time is the evidence of order.

## Why
The outside audit of the 17 strata dated 2026-09-26 (the first extended hourly run) found
false or unsupported claims on many of them (log: `memory/log.d/2026-09-27T0200Z-...`). The
human asked whether that is unusual. Nothing had ever been read this way, so there is no
baseline. This builds one.

## Samples (fixed now)
`sample.py`, seed 20260927: 8 strata drawn uniformly from each of four date windows of the
`date:` field (non-draft): June, July, August, 1-24 September 2026 (before hourly running).
32 layers. The drawn list is `sample.json`. No substitutions: a layer that turns out to make no
checkable claims is audited anyway and counts as clean, as Collatz-like layers did on 09-26.

## Procedure (fixed now)
The same brief as 09-26 (`oversight/claims-pass.md`), four layers per agent, eight agents,
agents not told the hypothesis or that this is a comparison. Findings are then classified by a
separate agent that sees only the findings and the rule below, not the cohort or the question.

## Classification rule (fixed now, applied to 09-26 below before any control data exists)
- **REAL-FACT**: a claim about the world contradicted by its primary source or by the page's
  own data, including misattribution (words or a date given to the wrong person or work) and
  a quantifier the data contradicts ("most", "tightest"); and the error would change what a
  careful reader believes about the subject. Last-digit rounding is not REAL-FACT.
- **REAL-READER**: something the page tells a reader to fetch or run that fails as written.
- **MINOR**: everything else (claims about the page's own checks, scope overstatements,
  unverifiable items, cosmetic inconsistency).

Primary metric: share of layers with at least one REAL-FACT. Secondary: REAL-READER share.

### 09-26 under this rule (classified now, from the reports already in hand)
REAL-FACT (8 of 17, 47%): the-hours-a-glass-ball-could-burn (Stokes/Scott; Scotland),
before-the-moon-comes-up (181 vs 101 sites), how-far-can-you-see-a-lighthouse ("most"),
how-many-points-it-takes (Bouts tightest; Panofsky as Kern), how-euripides-plays-are-dated
(Murray's date), phossy-jaw ("about" vs "less than" 5%; one death vs several),
eight-minutes-of-arc (Horizons reason), where-did-the-endurance-sink (one slip vs two).
REAL-READER (2 of 17): eight-minutes-of-arc (zip needs cd), how-long-do-ai-models-last
(pre-registration not reachable).
MINOR only: kelvin-age-of-the-earth (120 vs 121), loschmidts-paradox. Clean: 6.

## Guesses, written before looking
- G1: the control windows' REAL-FACT share is between 25% and 55% pooled, i.e. 09-26 is not
  an outlier. (If pooled controls are under 20%, hourly running is implicated.)
- G2: no clear trend by month.
- G3 (the human's question, measured separately from git in `maintain-rate.mjs`): the number
  of edits to already-published layers per layer per week has fallen as the corpus grew,
  while the absolute weekly count has not.

## Limits stated now
n=8 per window gives wide intervals (a 40% rate has a 95% interval of roughly 15-72%). Older
layers have had longer to be corrected, so a lower old-window rate can mean better birth or
more maintenance; each REAL-FACT will be checked in git for whether the wrong text was there
at the layer's first commit.