The Bias in the Sample

It has a name — Berkson's paradox, or in the modern idiom collider bias — and it is the third way a group statistic can lie while every underlying number stays honest. Its siblings on this site reclassify (stage migration) and aggregate (Simpson's paradox). This one selects: it shows that the simple act of choosing whom to include can forge an association out of pure independence. We build it three ways below, and recompute every number in front of you.

The intuition that a fourfold table drawn from hospital data can be analyzed as though it were a sample from the general population is erroneous. — Joseph Berkson (1946), paraphrased from the paper that named the effect

I · Everyone in the room has a reason

Berkson's original setting was a hospital. Take two conditions — call them A and B — that are completely independent in the city outside: knowing someone has A tells you nothing about whether they have B. The odds ratio between them, in the general population, is exactly 1. Now look only at the people who were admitted. If having either condition makes admission more likely, then among the admitted the two conditions become negatively associated — because almost everyone in a hospital bed has some reason to be there, and having one condition makes a second one less necessary to explain the admission.

Dial the prevalences and the admission rates below. The population odds ratio stays pinned at 1; the hospital odds ratio is computed exactly and printed beside it.

The exact identity is the surprising part, and the page checks it live: the odds ratio among the admitted equals (s₁₁·s₀₀)/(s₁₀·s₀₁) — the odds ratio of the admission rule itself — and it does not depend on the prevalences at all. Press shuffle prevalences and watch the two diseases get rarer or commoner while the hospital's verdict on them never moves. The association was never in the patients. It was in the door.

II · The same trick, with a dial

Berkson's hospital is the binary case. The continuous version is the one you feel every day. Give everyone two scores — quality X and quality Y — drawn independently from the same bell curve. In the full population their correlation is zero, dead flat. Now keep only the people whose two scores add up to clear some bar (the way a dating pool, a hiring shortlist, or a hall of fame admits you for looks or talent, food or ambiance). Inside that selected set, X and Y are anticorrelated — and the page computes by how much.

At the median cut — keep the top half by the sum — the invented correlation is not some messy empirical number but an exact constant: −1/(π−1) ≈ −0.4669. Tighten the bar and the phantom tradeoff sharpens: about −0.71 at the top 10%, −0.82 at the top 1%. The pickier the room, the more loudly it insists you can't have both. (Derived in closed form and cross-checked three ways — closed form, numerical integration, and a six-million-draw Monte-Carlo — in /research/berksons-paradox/.)

III · The one that mattered

This would be a parlour trick if it stopped at dating. It doesn't. For decades, data showed something that sounds like an argument for smoking while pregnant: among low-birth-weight babies, those whose mothers smoked died less often than those whose mothers didn't. The crossover is real and was replicated for fifty years. It is also a collider artifact — and here is the mechanism, run live.

Birth weight is a common effect of two things: smoking, and a bundle of other causes of a small baby (a serious birth defect, say) that are far deadlier than smoking. So restricting to small babies is selecting on "smoking or the deadly causes" — Berkson's bar again. Among the small, smokers' babies tend to be small for the milder reason, and so do better. Watch the two mortality curves cross.

Overall, smoking raises infant mortality — exactly as you'd fear. Slice to the small babies and it appears to lower it. Both are computed live from the same simulated cohort; nothing about any baby changed between the two readings. The only thing that changed is the box you drew around the data. This is a model of the mechanism, deliberately simple; the real, replicated crossover and its collider reading are cited in §VI.

…

IV · A collider, not a confounder

Here is the part that makes this dangerous rather than merely cute, and it is the exact opposite of the lesson Simpson's paradox teaches. A confounder is a common cause of two things — and you fix the bias it makes by controlling for it. A collider is a common effect — birth weight, hospital admission, making the dating pool — and conditioning on it is what creates the bias. Adjust for a confounder and you cure the problem; adjust for a collider and you cause it. In one line: confounding comes from a shared cause and is fixed by conditioning; collider bias comes from a shared effect and is fixed by not conditioning.

And the cruel detail: you cannot tell which you're looking at from the numbers alone. Whether a variable is a confounder you should adjust for or a collider you must leave alone is a fact about the causal structure of the world, not about the dataset. That is why "we controlled for it" is not automatically a virtue. Sometimes controlling for the wrong thing is precisely how the lie gets in.

V · The third way grouping lies

This page completes a trilogy. Each entry takes a statistic where every individual number is correct and the conclusion is still wrong — and each fails in a different place.

And the same trap is everywhere a sample was chosen by something the traits affect:

VI · Sources