Average enough of almost any randomness and you get the same bell curve — no matter what you started from. It is the most useful fact in statistics. It is also not always true, and the exception is hiding in plain sight.
Roll one die and the outcomes are flat — a 1 is as likely as a 6. Roll two and add them, and the flatness is already gone: a 7 comes up six times as often as a 2. Keep adding dice and the histogram of the sum climbs toward a shape nobody put there on purpose — the bell curve, the Gaussian, the normal distribution. Start from a different randomness — coin flips, waiting times, a lumpy two-valued thing — and you land on the same bell. This is the central limit theorem, and its strangeness is the sameness: the destination forgets where the journey began.
The oldest way to see it is a board of pegs. Drop a bead; at each row it goes left or right on a fair coin; it lands in a bin. Do it thousands of times and the pile is a bell — Abraham de Moivre worked out its exact shape in 1733, as the limit of the coin-flip (binomial) counts. Drop some beads:
beads dropped 0 · rows 12 · bins each L/R on a fair coin — the pile you build is the exact binomial C(n,k)/2ⁿ (the curve overlaid)
The smooth line is de Moivre's normal approximation N(mean = n/2, variance = n/4). The bars are your beads. They converge; the more you drop, the tighter the agreement.
The board is one machine. The theorem is the claim that almost every machine ends here. To feel how little the starting shape matters, pick a source of randomness — a flat one, a lopsided one, a two-humped one that looks nothing like a bell — and watch what the average of n draws does as you turn n up.
draws averaged 0
this histogram's mean —
→ predicted μ — ·
spread sd —
→ predicted σ/√n —
The frame is held fixed at the raw source's ±4σ. At n = 1 you see the source itself — flat, lopsided, or two spikes. Turn n up and the same bell forms and narrows: the width is σ/√n, so four times the sample halves the spread. The two-spike source has no bell in it at all, yet by n = 30 it has one.
A sum smears. Add two independent things and the result's distribution is the two convolved — every bump in one is dragged across every bump in the other, and sharp features blur. Do it again and again and the only shape that survives is the one that reproduces itself under this smearing while keeping a finite spread: the Gaussian is the fixed point. The theorem's one real requirement is that each thing you add has a finite variance — a finite typical size for how far it strays. Almost everything does. So the bell is almost everywhere, and it is tempting to believe it is everywhere.
It isn't. There is a distribution that draws you in with a familiar, symmetric hump — and then refuses the theorem entirely.
samples 0 ·
running average of Normal(0,1) — ·
running average of Cauchy —
biggest single Cauchy draw so far —
— every so often a monster arrives and yanks the whole average with it
Both lines are a running average, extended left to right as samples arrive. The teal line (a normal source) homes in on 0 and stays — that is the law of large numbers doing its job. The rust line (a Cauchy source) never settles: it lurches, forever, because a Cauchy draw has no typical size. Averaging a thousand of them is no calmer than a single one.
The Cauchy looks tame — a bell-ish hump, symmetric about zero. But its tails fall off so slowly that the integral defining its variance diverges: there is no finite typical size for how far a draw strays. Truncate it at ±T and the second moment grows like T without bound. The central limit theorem's one hypothesis is exactly the thing Cauchy doesn't have.
And the failure is total, not gradual. The average of n independent standard Cauchy draws is itself a standard Cauchy — provably, exactly. (Its characteristic function is e−|t|; the average's is [e−|t|/n]ⁿ = e−|t| — unchanged.) So the spread of the average never shrinks. We measured it offline, seeded and reproducible, at four sample sizes:
| average of n draws | median | interquartile spread |
|---|---|---|
| n = 1 | −0.008 | 2.00 |
| n = 10 | −0.004 | 2.00 |
| n = 100 | −0.002 | 2.00 |
| n = 1000 | −0.006 | 2.00 |
Read the last column. For a flat source, that spread falls by √1000 ≈ 32× from the first row to the last. For Cauchy it does not move. You can collect a thousand measurements and be exactly as uncertain as you were after one. You cannot average out the noise.
"Take more measurements and average" is the reflex of every lab and every poll, and it works because the central limit theorem is almost always true: the error bar shrinks like 1/√n. But the reflex quietly assumes finite variance, and the world's most consequential quantities often break that assumption — heavy-tailed things where one rare event dwarfs everything before it: sizes of cities and earthquakes, insurance losses, market crashes, wealth. Average a heavy-tailed sample and the mean is dominated by the largest draw you happened to see; collect more data and a bigger draw eventually arrives and moves it again. The sample never converges to a stable answer, because there may be no stable answer to converge to.
Two honest cautions, since both are common errors:
· The theorem is about sums and averages, not raw data. "My data is bell-shaped" is a separate claim, and often false — heights are roughly normal, incomes emphatically are not. What the theorem promises is that the average of many incomes behaves normally (if their variance is finite), never that any one income does.
· Convergence has a speed, and skew slows it. The Berry–Esseen theorem bounds how far the sum's distribution can be from the bell: the gap shrinks like 1/√n, times the source's skewness. A lopsided source (the "Lopsided" button above is an exponential) still looks lopsided at n = 1 and only straightens out slowly. The bell is a limit; at any finite n it is an approximation, and the tails are the last part to arrive.
Nothing on this page is asserted; it is recomputed. The Galton board's overlay is the exact binomial C(n,k)/2ⁿ by integer arithmetic, converging to de Moivre's normal peak 1/√(2πσ²). The three sources' sample means match μ and σ²/n to under 4% by seeded Monte Carlo at n = 1, 2, 5, 30. The normal CDF is built from scratch (68% within 1σ, 95% within 1.96σ). The Cauchy table above is reproduced with a fixed seed: median ≈ 0, IQR ≈ 2.00, unchanged from n = 1 to n = 1000 — while a finite-variance source's spread shrinks by exactly √n over the same range.
Full offline verifier: research/central-limit/verify.mjs — 56 / 56 checks pass, no dependencies, seeded so it reproduces bit-for-bit.