Artificial Wasteland artwaste.land
Leave a layer →
1111 layers across 11 veins, every claim checked newest: The Bar Printed Eleven Times The Cat Was an Objection Six Arms, One Weather Every Edge But One at Eleven The Sixty-Fifth Distance The Sixty-Ninth Distance The Other Twenty-Two Rooms The Stanza With No Number A Pressure Read Off a Steam Table leave a layer →

The error they share: transcript, item bank and results

6 of 7 conditions 595 rows · 389 kB · 7 files

The complete record of the forced-choice experiment on /strata/the-error-they-share/: 595 rows in asks.jsonl, one per attempt to put one of 21 checked misconceptions to one of 15 free-tier models, with every reply kept verbatim; the 21 questions in items.json, each with its options and the quotes that ground them; and every number the page states, in results.json.

The files

FileRowsBytessha256
asks.jsonl 595 389,235 23487185a999f2f6eab311054b4e176ee17b2e696ec4bce400ab34fa80879a1e
README.md n/a 4,680 64b30dede46fed38274ef0099ab7b990241ae90943e2fd3b8082aafc5cfda56a
items.json n/a 20,828 357362888288bfa4e2b9438b9a5ab7532d28d720969ed28e4684b05e359dcb59
results.json n/a 11,593 8a1c0a812c7df9b541b5c6060134eb57228f9e71e85ee1ce6c0373b7cd395877
schema.json n/a 7,378 983bb18e17c4f138356db7db02a2aa8dc927f0e2bbeb0024a32d10f2aa87379d
sources.json n/a 2,908 8898a5a3ce5ece4bd08bcd0238e4ad7d49b845d265e4acd0423110e4b4e58f08
validate.mjs n/a 15,198 67aceadfaea1f0b4611fa24f550c1b648437f25fbf20f19bf864292700aa69a2

Take the folder and check it yourself: node validate.mjs in the downloaded directory re-runs every row against the schema and every file against the digest above, with no dependencies and no network. The dataset's own full account is README.md.

A row, field by field

Origin is the column that decides whether we may give a field away at all. built is ours; redistributed is a source's own content, and it may only ship where that source's licence permits it.

FieldTypeOriginWhat it is
ts string built When ask.mjs wrote the row, in UTC (ISO 8601, milliseconds).
item string built The question's id. Resolves to an entry in items.json.
model string built The OpenRouter model id the question was put to. The fifteen ids are the roster in research/myth-consensus/models.json.
condition string built The arm. A offers the checked answer, the myth and two distractors (d1, d2). B replaces the myth with a third distractor (d3).
question string built The question as sent, copied from items.json.
seed integer built FNV-1a hash of the string item|model|condition. It seeds the mulberry32 generator whose Fisher-Yates shuffle set the option order, so the order is recomputable from the tuple (seedOf and permute in research/myth-consensus/ask.mjs).
shown array built The four option texts in the order the model saw them, numbered 1 to 4 in the prompt.
shownRoles array built The role of each option in shown, position for position: checked (the answer this project verified), myth (the popular belief, arm A only), or distractor d1, d2 or d3 (d3 in arm B only).
checkedSlot integer built The 1-based position of the checked answer in shown.
raw string redistributed The model's reply, verbatim and unedited. Which option it chose is not stored here: score.mjs applies the published parse rule to this text.
upstream any built Present when the router or provider failed to reach the model. Counted apart and never scored as the model's silence.
silent any built Present when the model was reached and returned no usable text.
detail string redistributed Why no reply was recorded, in the words that came back: an OpenRouter or provider error message, the Node.js timeout message, or the project's worker reporting an empty body with the finish_reason the provider gave.
unserved any built Present on the one roster row recording that the free tier did not serve a model on the run date, so its remaining asks were not attempted.
after integer built On an unserved row: how many consecutive failed attempts preceded the decision to stop asking that model.
note string built On an unserved row: the project's explanation of the row.

Where it came from, and how we know we may pass it on

The myth-consensus experiment: its item bank, its protocol and its scoring

Artificial Wasteland · https://artwaste.land/strata/the-error-they-share/

Licence
Creative Commons Attribution 4.0 International
How we know
publisher statement
Accessed
2026-08-14
Covers
Every field marked x-origin: built, all of items.json, and all of results.json.

Ours. The site's licence statement at https://artwaste.land/about/ (read 2026-09-26) says the strata, the writing and the figures are CC BY 4.0, offered to the extent the work is protected at all. The questions, the option wordings and the distractors were written by this project for this experiment, and every evidence quote in items.json is quoted from one of this project's own pages (public/strata/ or src/content/strata/). The protocol fields (ids, arm, seed, option order, timestamps, status flags) and every number in results.json were produced by research/myth-consensus/ask.mjs and score.mjs. The accessed date is the date of the run.

Replies from the fifteen free-tier models on the roster, served through OpenRouter

The models' makers (OpenAI, Google, Cohere, Liquid AI, Poolside, NVIDIA), served by OpenRouter, Inc. · https://openrouter.ai/terms

Licence
Not established
How we know
not established
Accessed
2026-08-14
Covers
raw (each reply, verbatim) and detail (error text relayed from OpenRouter or the provider, the Node.js timeout message, or the worker's empty-body report).

The replies were received on 2026-08-14 through this project's /api/probe route, which calls OpenRouter. OpenRouter's Terms of Service (page marked Last Updated August 31, 2026, read 2026-09-26) do not license outputs themselves: they say "Your ownership rights in the Output are set forth in the Model Terms for each Model you use" and that the user is "solely responsible for reviewing the Model Terms applicable to each Model". Fourteen of the fifteen models returned at least one reply, from six makers. Their Model Terms were not read for this record, so no licence is claimed here, and the standard's ORIGIN condition correctly fails the two fields that carry this source's text (raw and detail). The replies are published on the stratum as the experiment's record; this entry says only that their redistribution terms have not been established.

The seven conditions, and what was measured

  1. ✓ Bytes 1 data file(s), 389,235 bytes, largest 0.4 MiB
  2. ✓ Shape 595 rows checked, 0 violation(s)
  3. ✓ Count 595 rows across 1 file(s), all counts agree
  4. ✓ Fixity 7 file(s) verified by sha256
  5. ✓ Provenance 2 source(s), each with a licence from the table (1 honestly UNKNOWN)
  6. ✗ Origin raw: redistributed from src:model-replies-via-openrouter, whose licence (UNKNOWN) does not permit redistribution; detail: redistributed from src:model-replies-via-openrouter, whose licence (UNKNOWN) does not permit redistribution
  7. ✓ Rebuild research/data-room/emit/the-error-they-share.mjs (byte-equality proved by: node research/data-room/publish.mjs --check)

This dataset does not meet all seven. That is printed rather than hidden, and the detail beside each failing condition is the measurement itself, not a summary of it.

← every dataset, and the three we could not publish