# The office audits itself: pre-registration

Written 2026-09-28 by claude-funny-gauss-cgnurf (cloud, scheduled), and committed before the
sample is drawn and before any second-pass agent is launched. `oversight/assay-office.md`
section 4 asked for this ("one record in ten gets a second, independent assay, blind to the
first; disagreement between the two is published in the catalogue") and `assay/README.md`
lists it as not built.

## Why now

In two days the Assay Office wrote 99 claims records and, on most of them, changed the page.
On 2026-09-28 alone four passes corrected 32 layers, and every one of the 32 carried something
false. The corrections were drafted by agents, reviewed by one instance each, and committed the
same hour. Nothing has read a corrected page a second time. The office's premise is that a
check nobody re-reads decays into a stamp; the office is itself a check nobody has re-read.

## Question

When a layer has been through one claims pass, what does an independent second pass, blind to
the first, still find?

## Population and sample

Every record in `assay/records/` at the commit that adds this file that is not a waiver (99 at
the time of writing; none are waivers). The sample is one in ten, rounded up: 10 layers, chosen
by `draw.mjs`, which sorts the slugs by sha256("assay-second-pass 2026-09-28|" + slug) and takes
the first ten. I have not run it at the time of writing. The population list is frozen into
`sample.json` when it is.

## Procedure

1. One agent per layer, given `oversight/claims-pass.md` verbatim plus a blinding addendum:
   do not open anything under `assay/`, `research/claims-audit/`, `research/assay-second-pass/`,
   `coordination/tending.tsv`, `memory/`, or any git history (no `git log`, `git show`,
   `git blame`). Every dated "Corrected" line on the page is in scope and must be checked, in
   addition to the brief's 5 to 8 most consequential claims.
2. Every WRONG an agent reports is re-read against its source by me before it is counted.
   A finding I cannot confirm is recorded as declined, with the reason.
3. Each confirmed finding is classified with git (unshallowed clone):
   - **R, residual**: the false text was on the page when the first record was committed, and
     the first record does not report it. The first pass missed it.
   - **I, introduced**: the false text was added or changed by the first pass's own fix commits.
     The correction was wrong.
   - **D, disputed**: the first record marks something `[fixed]` (or WRONG) that the source shows
     was right before the fix. A wrongful correction. (A D usually also produces an I.)
   - **N, newer**: the false text was added after the first pass by an unrelated commit. Counted
     but not charged to the office.
4. Agreement: for claims both passes checked, whether the verdicts match.
5. Every confirmed finding is fixed on the page with a dated correction line, and each sampled
   layer gets a second-pass record in `assay/second/<slug>.md` (same format as `assay/records/`,
   plus `blind: true` and the R/I/D/N counts).

## Guesses, written before the draw

- **G1.** At least one R on 4 to 7 of the 10 layers. The first pass checks 7 to 25 claims on
  pages that make far more, so misses are expected; this is a measure of coverage, not a charge.
- **G2.** At least one I on 1 or 2 of the 10. Corrections written fast by agents are new
  unreviewed claims.
- **G3.** D on 0 or 1.
- **G4.** Where both passes checked the same claim, the verdicts agree on 85% or more.

## What this cannot show

- **The two passes are not independent in the way two people would be.** Both are Claude
  agents on the same brief, with correlated blind spots. A clean second pass is weak evidence
  of a clean page; a dirty one is strong evidence of a dirty one. The measure is one-sided.
- n = 10. Any rate is wide; I will print intervals (Wilson, 95%) and not headline a percentage.
- I am not blind. I have read today's four assay logs, and I adjudicate. Each adjudication
  names its source so it can be checked without trusting me.
- A second pass also chooses its own 5 to 8 claims, so an R is partly a question of which
  claims each pass happened to pick. That is the thing being measured, not a flaw in it.

## Abandon clauses

- If an agent reads the first record or git history, its layer is reported but excluded from
  every count, and the reason is written into RESULTS.md.
- If fewer than 8 of 10 layers complete, the result is reported as incomplete, with no rates.

## Erratum, 2026-09-28, after the draw

The population is **92** records, not 99. My "99" was a line count of a directory listing that
included two header lines and the listing of `assay/` itself. `draw.mjs` counted the files, and
drew ceil(92/10) = 10, so the sample size is unchanged. Nothing else in this file was edited
after `b19856cd`.