{
  "slug": "thirty-seven-thousand-curtains",
  "title": "Thirty-Seven Thousand Curtains",
  "href": "/strata/thirty-seven-thousand-curtains/",
  "field": "psychology (experimental parapsychology, replication)",
  "claim": {
    "statement": "Across all 100 sessions, participants correctly identified the future position of the erotic pictures significantly more frequently than the 50% hit rate expected by chance: 53.1%, t(99) = 2.51, p = .01, d = 0.25.",
    "claimants": "Daryl J. Bem",
    "citation": "Daryl J. Bem (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, volume 100, issue 3, pages 407 to 425. DOI 10.1037/a0021524. PMID 21280961. Quoted from the version of record, p. 409.",
    "year": 2011,
    "headline_as_printed": "53.1%, t(99) = 2.51, p = .01, d = 0.25",
    "programme_as_printed": "The mean effect size (d) in psi performance across all 9 experiments was 0.22, and all but one of the experiments yielded statistically significant results."
  },
  "records": [
    {
      "role": "claimant-data",
      "name": "Bem (2011) Experiment 1, one row per session: erotic and nonerotic trials and hits, stimulus-seeking score (derived from the per-session percentages in exp1.xlsx; the workbook itself is not shipped)",
      "file": "data/bem-exp1-sessions.csv",
      "url": "https://replicationindex.com/wp-content/uploads/2018/01/exp1.xlsx",
      "licence": "No licence stated. Released by the author through the correspondence page (\"I hereby grant you permission to be the conduit for making my data available to those requesting them.\", Bem to Schimmack, 20 January 2018). Only a derived factual extract of counts and scores is shipped.",
      "licence_url": "https://replicationindex.com/2018/01/20/bemcorrespondence/",
      "retrieved": "2026-09-22",
      "sha256": "7512bdd237da6d77d823629f9f3b0a5b28b21801b24dff9ff435c17f277a7db0",
      "upstream_sha256": "feadd752a26f997327f12921eb916d9154b9907e27205aca64b9a22a0ea2df45",
      "units": 100
    },
    {
      "role": "control-data",
      "name": "Transparent Psi Project, every erotic trial surviving the three preregistered exclusions, in file order (participant index, trial number, guessed side, target side)",
      "file": "data/tpp-erotic-trials.csv",
      "url": "https://osf.io/download/24qkr/",
      "licence": "CC BY 4.0 (OSF node fa247 and its parents); fields extracted, participants re-indexed; planted copies are made in the browser and never shipped",
      "licence_url": "https://creativecommons.org/licenses/by/4.0/legalcode",
      "retrieved": "2026-09-22",
      "sha256": "74cefeeb072076bef29e8e6f81bd622dbbe2378a5c2cfd961b5aeae21a0bc1b7",
      "upstream_sha256": "2389f4f82ff48b1609d3f7352e7967111bf741cb8674570925bbe791892a7b4f",
      "units": 38095
    },
    {
      "role": "control-data",
      "name": "Transparent Psi Project participants: laboratory index and the two sensation-seeking item codes from each participant’s first erotic row",
      "file": "data/tpp-participants.csv",
      "url": "https://osf.io/download/24qkr/",
      "licence": "CC BY 4.0; derived fields only (no user agents, ages, sexes or identifiers)",
      "licence_url": "https://creativecommons.org/licenses/by/4.0/legalcode",
      "retrieved": "2026-09-22",
      "sha256": "e1dc49a9dc9c769e2fbd459991c3b40f15c45f38e587b91726d38ae38679518a",
      "upstream_sha256": "2389f4f82ff48b1609d3f7352e7967111bf741cb8674570925bbe791892a7b4f",
      "units": 2128
    },
    {
      "role": "reference",
      "name": "Laboratory labels from table 1 of Kekecs et al. (2023), matched by participant and erotic-trial counts; the two added test-account IDs identified by prefix",
      "file": "data/tpp-labs.json",
      "url": "https://doi.org/10.1098/rsos.191375",
      "licence": "CC BY 4.0 (article)",
      "licence_url": "https://creativecommons.org/licenses/by/4.0/",
      "retrieved": "2026-09-22",
      "sha256": "5500f024eb2bcbd316a144d95160aac6739cb186065c97565ba0bc266f0322de",
      "units": 12
    },
    {
      "role": "reference",
      "name": "Independent recomputation in Python and scipy from the two upstream files (research/thirty-seven-thousand-curtains/crosscheck.py)",
      "file": "research/thirty-seven-thousand-curtains/data/crosscheck.json",
      "url": "https://osf.io/download/24qkr/",
      "licence": "derived numbers, this site",
      "licence_url": "https://creativecommons.org/licenses/by/4.0/legalcode",
      "retrieved": "2026-09-22",
      "sha256": "55d4e80b28c9c9baad6b9bb5b68c03a5c67d9efd57e848146b8283761edf13a7",
      "units": 1
    }
  ],
  "anchor": {
    "what": "Bem’s one-sample, one-tailed t test of the 100 session erotic hit percentages against 50 (engine.sessionTTest), and his pooled binomial test with a continuity correction, run on the per-session extract of his workbook.",
    "expected": "53.1%, t(99) = 2.51, p = .01, d = 0.25; binomial z = 2.30, p = .011",
    "computed": "53.139%, t(99) = 2.5133, p = 0.0068, d = 0.25; 826 of 1,560, z = 2.30, exact p = 0.0106",
    "agrees": true,
    "tolerance": "Printed rounding: the hit rate must round to 53.1 at one decimal, t, d and z must lie within 0.005 of the printed two-decimal values, p must round to the printed digits. The erotic-minus-nonerotic t computes to 1.8563 against a printed 1.85, a last-digit gap shown on the page. No tolerance is needed for digitisation: the numbers come from the author’s released workbook.",
    "note": "The printed 53.1% is the mean of session percentages; pooled over trials it is 826 of 1,560 (52.95%). The replication used 828 of 1,560."
  },
  "control": {
    "name": "The Transparent Psi Project: a preregistered, consensus-designed replication of Bem’s Experiment 1, decided by four tests (random-intercept logistic regression with a Bonferroni-widened Wald interval, and three beta-prior Bayes factors) at five registered looks",
    "citation": "Zoltan Kekecs, Bence Palfi, Barnabas Szaszi, Peter Szecsi, Mark Zrubka, Marton Kovacs, Bence E. Bakos, Denis Cousineau, Patrizio Tressoldi, Kathleen Schmidt, Massimo Grassi, Thomas Rhys Evans, Yuki Yamada, Jeremy K. Miller, Huanxu Liu, Fumiya Yonemitsu, Dmitrii Dubrov, Jan Philipp Röer, Marvin Becker, Roxane Schnepper, Atsunori Ariga, Patrícia Arriaga, Raquel Oliveira, Nele Põldver, Kairi Kreegipuu, Braeden Hall, Sera Wiechert, Bruno Verschuere, Kyra Girán and Balazs Aczel (2023). Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project. Royal Society Open Science 10(2), 191375. doi:10.1098/rsos.191375. With: Correction, Royal Society Open Science 10, 231080 (16 August 2023), doi:10.1098/rsos.231080.",
    "computed": "18,876 hits in 37,836 erotic trials (49.889%) from 2,115 participants; mixed-model 99.75% interval 49.107% to 50.671%; BF01 182.1 (replication prior), 212.3 (uniform), 72.4 (BUJ); all four tests M0 at the first registered look, so the rule stops for M0",
    "as_printed": "49.89% successful guesses within 37,836 erotic trials, 2,115 participants; 99.75% CI 49.11% to 50.67%; 72 times more likely at random (in a conclusion pre-written during the consensus design, which does not name the prior; of the three Bayes factors only the BUJ one rounds to 72); all four primary tests supported M0"
  },
  "power": {
    "grade": "A-injection",
    "planted": "Bem’s printed 53.1%, planted into a copy of the first 37,836 replication erotic trials: 1,215 misses chosen by a seeded Fisher and Yates shuffle (mulberry32, seed 20110131) have their target side set to the guessed side, for 20,091 hits; order, participants, trial numbers and guesses unchanged",
    "recovered": "The unmodified four-test rule on the copy: interval 52.32% to 53.88%, log10 BF01 −30.9 (replication), −29.7 (uniform), −30.2 (BUJ); all four M1, the rule stops for M1 at the first look. The real trials through the same function stop for M0.",
    "verdict": "the control COULD have confirmed the claim",
    "also": "828 of 1,560 (+1,206) and 826 of 1,560 (+1,158) also stop for M1; 51.0% (+420) is not settled at the first look (the mixed model alone says M1, the uniform BF01 is 0.041); 57.6% planted into the stimulus seekers only (+1,226) stops for M1.",
    "scope": "The plant works at the level of the released trials: it trusts that the replication’s software recorded guesses and targets as it says, and it cannot test stimuli, rooms or experimenters. It shows that the registered rule detects a 53.1% hit rate in these trials, not that an effect real under Bem’s conditions would have reached that size with the replication’s different erotic images and shared testing spaces."
  },
  "null_through_claimant_method": {
    "applicable": true,
    "what": "Bem’s session t test (the anchor’s own sessionTTest) on 10,000 seeded null experiments with his exact structure (40 sessions of 12 fair erotic trials, 60 of 18; seed 409), and on the replication’s own first 2,100 participants cut into 21 blocks of 100 (the control’s data, not known to be empty).",
    "result": "t of at least 2.51 in 60 of 10,000 null runs (0.60%; the t distribution gives 0.68%); one-tailed p below .05 in 498 (4.98%). 0 of 21 replication blocks reach p below .05 (t from −2.16 to 1.23). The procedure is calibrated; it does not manufacture the claim.",
    "yardstick": "The replication’s Bayes factors on Bem’s 826 of 1,560: BF10 0.95 (uniform), 2.71 (BUJ), inside the inconclusive band; the replication prior is refused on Bem’s own data."
  },
  "verdict": {
    "category": "DISSOLVED",
    "as_of": "2026-09-22",
    "sources": [
      "Zoltan Kekecs, Bence Palfi, Barnabas Szaszi, Peter Szecsi, Mark Zrubka, Marton Kovacs, Bence E. Bakos, Denis Cousineau, Patrizio Tressoldi, Kathleen Schmidt, Massimo Grassi, Thomas Rhys Evans, Yuki Yamada, Jeremy K. Miller, Huanxu Liu, Fumiya Yonemitsu, Dmitrii Dubrov, Jan Philipp Röer, Marvin Becker, Roxane Schnepper, Atsunori Ariga, Patrícia Arriaga, Raquel Oliveira, Nele Põldver, Kairi Kreegipuu, Braeden Hall, Sera Wiechert, Bruno Verschuere, Kyra Girán and Balazs Aczel (2023). Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project. Royal Society Open Science 10(2), 191375. doi:10.1098/rsos.191375. With: Correction, Royal Society Open Science 10, 231080 (16 August 2023), doi:10.1098/rsos.231080.",
      "Jan Walleczek, Nikolaus von Stillfried, Stefan Schmidt, Marc Wittmann, Karolina A. Kirmse, Jorge Moll and Zoltan Kekecs (2025). Metascientific replication project with the advanced meta-experimental protocol of the transparent psi project procedures for testing the precognitive effect claimed by Bem. PLOS ONE 20(11), e0335330, published 5 November 2025. doi:10.1371/journal.pone.0335330.",
      "Michael Duggan. Transparent Psi Project. Psi Encyclopedia, Society for Psychical Research; created 7 February 2023, last updated 9 July 2026. https://psi-encyclopedia.spr.ac.uk/articles/transparent-psi-project/"
    ],
    "scope": "An above-chance hit rate on erotic trials in the Experiment 1 paradigm of Bem (2011). Nothing about the other eight experiments of that article, or about ESP in general.",
    "would_change": "A preregistered replication of this paradigm, with the same transparency, that the replication’s own four-test rule (or one as strict) settles for M1; or a moderator present in Bem’s Cornell sessions and absent from the replication (selected participants, an altered state, a stimulus set), shown prospectively to produce above-chance hit rates."
  },
  "decided_by": "independent-replication",
  "years_to_decision": 12,
  "choices": [
    {
      "id": "claimed_size",
      "label": "Claimed size to plant",
      "options": [
        "printed 53.1%",
        "828/1560",
        "826/1560",
        "51.0%",
        "custom"
      ],
      "default": "printed 53.1%",
      "affects": [
        "planted hits",
        "three BF01",
        "mixed-model interval",
        "look-1 decision"
      ],
      "inert": false
    },
    {
      "id": "plant_shape",
      "label": "Plant shape",
      "options": [
        "everyone",
        "stimulus seekers at 57.6%"
      ],
      "default": "everyone",
      "affects": [
        "random-intercept SD",
        "mixed-model interval",
        "planted hits"
      ],
      "inert": false
    },
    {
      "id": "rule",
      "label": "Which tests decide",
      "options": [
        "registered four",
        "Bayes factors only",
        "mixed model only"
      ],
      "default": "registered four",
      "affects": [
        "look-1 decision for a 51.0% plant",
        "operating characteristic"
      ],
      "inert": false,
      "note": "The \"Bayes factors only\" option gives the same decision as the registered four everywhere on the page, because under the registered rule the mixed model never binds (its thresholds lie inside the Bayes factors’ at all five looks); the page says so beside the option and the verifier asserts it. \"Mixed model only\" moves both named outputs."
    },
    {
      "id": "exclusion",
      "label": "Exclusion list",
      "options": [
        "as run (five IDs)",
        "as preregistered (three IDs)"
      ],
      "default": "as run (five IDs)",
      "affects": [
        "hits",
        "hit rate",
        "participants",
        "three BF01"
      ],
      "inert": false,
      "note": "The verdict does not move."
    },
    {
      "id": "ss_cut",
      "label": "Stimulus-seeking cut",
      "options": [
        "at least 3.0",
        "more than 3.0",
        "any half point from 1.0 to 5.0, at least or more than"
      ],
      "default": "at least 3.0",
      "affects": [
        "Bem subgroup",
        "replication subgroup"
      ],
      "inert": false
    },
    {
      "id": "bem_subset",
      "label": "Bem’s sessions",
      "options": [
        "all 100",
        "1 to 40",
        "41 to 100"
      ],
      "default": "all 100",
      "affects": [
        "session readout"
      ],
      "inert": false,
      "note": "The anchor always uses all 100."
    },
    {
      "id": "true_rate",
      "label": "True hit rate (operating characteristic)",
      "options": [
        49,
        50,
        50.7,
        51,
        53.1,
        53.5
      ],
      "default": 50.7,
      "affects": [
        "P(M1)",
        "P(M0)",
        "P(inconclusive)",
        "expected trials"
      ],
      "inert": false
    },
    {
      "id": "null_seed",
      "label": "Seed of the null experiments",
      "options": [
        409,
        "any 32-bit integer"
      ],
      "default": 409,
      "affects": [
        "null counts"
      ],
      "inert": false
    }
  ],
  "headline": "The replication Bem helped design found 49.89% in 37,836 erotic trials; planted into a copy of those same trials, his 53.1% makes the unmodified four-test rule stop for it at the first look, so the control could have confirmed the claim, and did not.",
  "novelty": "We searched the Transparent Psi Project article, its 2023 correction, its stage 1 protocol and supplement, the 2025 AMP-TPP article, the Psi Encyclopedia entry (last updated 9 July 2026), the Wagenmakers et al. 2012 confirmatory replication report, James E. Kennedy’s 2023 audit of the project and its companion lessons document, and web searches on 2026-09-22 and did not find a published test of Bem’s two-item stimulus-seeking moderator in the Transparent Psi Project’s released trials.",
  "spec_corrections": [
    "The 3.0-or-more stimulus-seeking group in the replication is 894 participants (15,997 trials, 7,988 hits), not 893, when scores are read as the replication’s own script reads them (the first erotic row of each participant); the spec’s 893 came from the first answered row of the whole file. Mean hit rate 49.94% either way; t = -0.15 against the spec’s -0.14.",
    "The operating characteristic’s far tails at 53.1% are P(M0) = 6.9e-30 and P(inconclusive) = 1.3e-79 by exact band propagation, against the spec’s FFT-based 7e-30 and 1e-31 (numerical noise at that scale).",
    "The paper’s printed target-left share, 49.88%, computes to 49.886% (49.89% rounded): a last-digit gap, shown."
  ],
  "ledger": {
    "claim_reproduced": {
      "value": true,
      "assertion": "anchor:bem-session-ttest-reproduces-53.1-t2.51"
    },
    "control_computed": {
      "value": true,
      "assertion": "control:tpp-four-tests-at-look-1"
    },
    "control_can_confirm": {
      "value": true,
      "assertion": "power:planted-53.1-recovered-as-M1"
    },
    "rendered_figures_asserted": {
      "value": true,
      "assertion": "surface:static-figures-and-head"
    },
    "choices_are_live": {
      "value": true,
      "assertion": "choices:each-changes-an-output"
    },
    "planted_fault_caught": {
      "value": true,
      "assertion": "fault:swapped-target-sides-and-18-trial-reading"
    },
    "refusal_asserted": {
      "value": true,
      "assertion": "refuse:below-observed-plant-and-replication-prior-on-bem"
    },
    "provenance_pinned": {
      "value": true,
      "assertion": "provenance:sha256-matches"
    },
    "verdict_dated": {
      "value": true,
      "assertion": "verdict:category-date-sources-printed"
    }
  }
}