# Deviations from PREREGISTRATION.md (part 1) Items 1 to 10 were decided before any TEST count of any repaired vocabulary existed and are in the commit history before the first measurement (the first, Llama 3's K = 100 arm, began at about 03:58 UTC on 30 October). Items 11 and 12 came after some TEST counts existed, and 13 and 14 after all of them; each says what it changed. None was chosen by looking at a result. 1. **WEB-TEST is capped at 1 MB per language.** The pre-registration took FineWeb-2's own `test` split whole where it is at least 100 KB. Those splits run to 100 MB (Danish) and would have filled the disk (the first 44 languages' splits came to 2.4 GB) and made every arm's measurement hours long. Now: seeded random row groups of the test split until at least 1.5 MB, then documents in ascending hash, capped at 1 MB; the hash-carved WEB-TEST (where the split is under 100 KB) is capped at 1 MB the same way; and "under 100 KB" is read from FineWeb-2's published size table (`data/fineweb2-language-distribution.csv`), not from the draw. English WEB-TEST (20 MB) and CODE-TEST (20 MB) are unchanged. 2. **Additions are single merges whose two parts already exist.** The pre-registration describes each candidate as a closure with its missing prerequisites, chosen by savings per slot. Evaluating multi-step closures as units over 204 languages at once was not tractable here, so a candidate becomes eligible once its parts are entries (whether original or chosen earlier), and is valued as one slot. Chains are built one step at a time, which is how BPE itself is trained. This can only miss savings (a chain whose first step is worth little alone), which works against H1 to H4. 3. **The main arm removes only leaves.** Of the entries the strict audit would allow, only those that are no component of any merge are candidates. Removing a leaf that VALUE never emits changes no VALUE encoding, which makes the guard exact and cheap; finding which pieces a non-leaf passes through would mean searching every VALUE piece for every candidate. The Didenko baseline keeps the full strict audit. 4. **A letter's cost is measured alone.** Letter candidates are code points that cost two or more tokens when encoded by themselves, not "in context". Their value is still measured in context, on VALUE, by selection. 5. **No merge that makes an entry V already has.** A pair whose concatenation is already an entry creates no new slot; such pairs are skipped by candidate generation and selection. 6. **Tie-breaks, which the pre-registration left open:** continued BPE takes the highest count, then the pair with the smaller (left, right) strings; selection takes the highest weighted saving, then the same. 7. **The two weighting sensitivities run at K = 1,000 only.** The pre-registration named them (equal bytes, and Common Crawl share with a floor of 1e-4) without fixing a budget. They use the same candidates, the same procedure and the same guards as the primary run, at K = 1,000 (H1's budget), with its APPEND arm and without the baselines or the Latin arm (`run_part1.py --weighting bytes|cc --Ks 1000`). Decided before any TEST count existed. 8. **Removal checks made exact (and fast).** The first `choose_removals` (a) skipped the guard for any leaf absent from a set of used ids computed once, before any removal, although an accepted removal can re-encode pieces so that a leaf absent before is used after; and (b) tried each later removal with the merges of earlier unused removals still in place. Either could only let a removal past the guard unchecked, never block one. The replacement keeps a sorted index of which pieces hold each id, adds the pieces each accepted removal re-encodes, and tries every removal with all earlier ones applied, which is what the pre-registration describes. Additions were also sped up (pair hits found from the flat pair codes, not a Python scan). On a 15-language subset of Llama 3's VALUE text, the new code chooses the same 300 additions and leaves identical encodings, and the same 800 removals with identical rises. Asked for 3,000 removals there, both accept the same 1,513 before the guard exhausts the candidates (61 minutes against 32 seconds), but the old code's tracked encodings and rises go stale; the new code's equal a fresh encoding by the final vocabulary on all 843,155 pieces, and no tracked encoding holds a removed id. Llama 3's run, then building its K = 1,000 arm with the old code, was stopped and rebuilt with the new; its K = 100 arms, which remove nothing, were kept. Decided 2026-10-30, before any TEST count. 9. **When guarded removals run out, an arm uses the slots it has.** Llama 3 has 6,090 removal candidates (leaves outside the protected set, deviation 3); its K = 5,000 arm needs 4,749 that each pass the guard. If the candidates run out first, the arm gets free ids plus the removals accepted, its additions are that many, and its APPEND arm and both baselines use the same number (the baselines remove that many). The achieved size is reported with every result. Decided 2026-10-30, before any TEST count. 10. **Base bytes are never removed.** Byte-level BPE starts every piece from one-byte entries, and the guard cannot see a missing one: BPE on the remaining merges still yields the byte, so counts do not change, but the written tokenizer can no longer encode it. Llama 3 has 23 one-byte leaves outside the protected set, o200k 18 and Qwen3 17; ten of Llama 3's (for example 0xF4, which starts every character of plane 16, and 0xDF, which starts N'Ko letters) and four of Qwen3's (0xEE, which starts most of the Private Use Area) can occur in valid UTF-8. Removal candidates (main arm and the Purason baseline; Didenko's already skips one-character entries) now exclude every one-byte entry except the 13 bytes UTF-8 never uses (0xC0, 0xC1, 0xF5 to 0xFF). Found while writing a test for the release tool, 2026-10-30; Llama 3's run had not yet reached a removal, and no TEST count existed. 11. **CODE-TEST was missing when Llama 3 was measured, and was rebuilt.** `data/fetch_code.py` wrote it on 2026-10-29 (20,000,099 bytes, 632 files, per its log), and it was deleted with the oversized FineWeb-2 test files during the disk clean-up behind deviation 1. Llama 3's run therefore measured CODE-TEST as empty (0 tokens for every arm), which leaves G1's code half untested, not passed. The file was rebuilt from the same pinned commits with the same script on 2026-10-30: 20,000,099 bytes, 632 files, sha256 a6153a821e2f7a4a02656f4784ad86a5d5d88e3b19bb52f4c0c6916cb6f049f4. `fix_code_test.py` then counts it for Llama 3's V and every frozen arm, and adds its round-trip check to G3; nothing about any arm changes. The gap was found on the first look at Llama 3's TEST counts (the code count read 0); o200k and Qwen3 are measured with the file in place. 12. **The baselines may not remove what the shared additions are built from.** Both baselines take the main arm's additions unchanged (the pre-registration's "they differ only in which entries are removed"). gpt-oss's Didenko baseline removed 13 entries that some of those additions are merges of, and its tokenizer could not be loaded; it was caught when the run tried to measure it, before any TEST count of that arm. Both baselines now skip every entry an addition is built from, as Didenko's own method requires an inserted entry's parts to exist. Llama 3's two baselines and gpt-oss's Purason baseline removed none of those entries, so they are unchanged; only gpt-oss's Didenko arm (and Qwen3's, not yet built) are affected. The tokenizer writer now refuses any merge whose parts are missing. Decided 2026-10-30. 13. **H5's second clause is now counted as registered.** The pre-registration says H5 needs "fewer languages' devtest counts rise at all under it than under either baseline"; `analyse_part1.py` had counted every TEST text that rose (FLORES languages and WEB-TEST slices together). Found by the claims pass (2026-10-30) and corrected to FLORES devtest languages; the text count is still reported. No verdict changed: H5 fails for all three vocabularies on its first clause (Purason's rule cut at least as much), and on the registered count Llama 3 has 31 languages rising against Purason's 30, gpt-oss 40 against 8, and Qwen3 1 against 2 and 13. 14. **Two errata in the frozen pre-registration** (it is stamped, so it is not edited). (a) Its "Seen" section says "H3 below is therefore not blind and is reported, not tested": a leftover from a draft in which H3 was the winner's curse, which the final text lists under "Reported, not tested"; H3 as registered (the first 500 slots' share) is a tested prediction in its predictions list and in `prereg.json`, and is tested as such. (b) `prereg.json` names Llama 3's open model as `meta-llama/Llama-3.2-1B`, and the text says "as used by Llama 3.x; 251 reserved special ids". The pinned tokenizer is Meta-Llama-3-8B's, with 251 reserved ids; Meta's current code for Llama 3.1 and later names five of them (`<|finetune_right_pad_id|>`, `<|step_id|>`, `<|eom_id|>`, `<|python_tag|>`, `<|image|>`). The repaired Llama 3 vocabularies fill those ids, so they are for Llama 3 itself, not 3.1 and later. Found by the claims pass, 2026-10-30.