Simpson’s Paradox Calculator

Berkeley, 1973 — the case that made it famous

The tool loads real data by default, not a constructed example. These are UC Berkeley’s graduate admissions figures for the six largest departments in autumn 1973, as published in Science in 1975.

Pooled, men were admitted at 44.52% and women at 30.35% — a gap of 14.16 percentage points, and on its face a serious finding.

Department by department, women had the higher admission rate in four of the six. In Department A, 82.4% of women were admitted against 62.1% of men. In Department B, 68.0% against 63.0%.

The explanation is where people applied, not how they were treated. Departments A and B admitted around 60% of applicants and received 1,385 male applications against 133 female. Department F admitted about 6% and received 341 female applications against 373 male. Women applied in far greater numbers to the departments that admitted almost nobody, so their pooled rate was dragged toward those departments’ base rates.

The original paper’s conclusion is the part usually lost. Bickel and colleagues did not conclude “no bias”. They concluded that the bias was not in the admissions decisions, and pointed at what puts women disproportionately into the most competitive fields in the first place. The paradox relocated the question rather than dismissing it.

The tool’s second preset is the other classic, from a 1986 comparison of kidney stone treatments. Treatment A wins on small stones (93.1% against 86.7%) and on large stones (73.0% against 68.8%), and loses overall (78.0% against 82.6%). That is a total reversal — every stratum one way, the pooled figure the other — because the more effective treatment was given to the harder cases.

It needs two things to be true at once

The reversal is not a fluke of small samples, and it is not a paradox in the logical sense. It is a property of weighted averages, and it requires exactly two conditions.

First, the strata must differ in their base rates. If every department admitted 40% of applicants, no distribution of applicants could move a pooled rate anywhere. The tool reports this spread — at Berkeley it is 58 percentage points between the easiest and hardest department.

Second, the two groups must be distributed differently across those strata. If men and women had applied to each department in the same proportions, pooling would be a fair weighted average with identical weights. The tool reports this spread too — at Berkeley it is 63 points.

The two conditions are not symmetric, and it is worth being exact. If both groups have the same rate in every stratum, no stratum has a direction and there is nothing to reverse. If the composition is identical across strata, the pooled difference is a positively-weighted average of the stratum differences, so it cannot contradict a unanimous verdict — a total reversal becomes impossible. A bare majority can still be outvoted by a single large stratum, so equal composition does not rule out every reversal, only the complete kind. The verification suite asserts exactly that distinction, because an earlier draft of this page claimed the stronger version.

This is why randomised experiments are immune. Randomisation forces the second condition to fail: the groups are distributed across every stratum in the same expected proportions, so composition cannot differ systematically. It is one of the sharpest arguments for randomisation there is, and it does not depend on any assumption about the outcome.

And it is why observational data is so exposed. In real data people sort themselves into strata for reasons connected to the outcome — sicker patients get the stronger treatment, harder applicants apply to harder departments — which produces both conditions at once, by the same mechanism.

Standardisation is the repair, not a caveat

Most treatments of this paradox stop at “be careful with pooled rates”. There is an actual calculation that fixes it, and the tool performs it.

Direct standardisation applies both groups’ own within-stratum rates to one shared set of weights. Each group keeps the rates it actually achieved; only the mix is equalised. The result answers “what would each group’s overall rate have been if they had been distributed the same way?” — which is the question a fairness comparison was always trying to ask.

On Berkeley it changes the answer. The raw gap is +14.16 points favouring men. After standardising it is −4.26 points, favouring women. An 18-point swing, and a change of sign, from nothing but the weighting.

The choice of weights is a real decision, and it should be stated. This tool uses the combined totals, which is the conventional choice when neither group is privileged as the standard. Using one group’s distribution instead gives a different number and answers a different question — “what would women’s rate be if they applied like men?” is not the same as “what would both be under a common standard?”

It only controls for the strata you have. Standardising by department removes department composition and nothing else. If another variable is doing the same work — and you did not measure it — the standardised figure is exposed to it exactly as the raw one was. This is the same limit that partial correlation runs into.

Which number is right depends on the question

The temptation is to declare the stratified figures “true” and the pooled one “misleading”. That is usually right and it is not automatic, and the difference matters.

Ask about the process and the strata win. “Does this department treat men and women differently?” is answered department by department. Pooling contaminates it with where people applied, which the admissions committee did not choose.

Ask about the outcome and the pooled figure is a fact. “What share of women who applied to Berkeley got in?” is 30.35%. That is not misleading; it is the answer. It matters for capacity planning and for describing what actually happened to a cohort.

The genuine rule is causal, not statistical. Condition on a variable when it is a common cause of both the group and the outcome. Do not condition on it when it sits on the causal path between them — if the field someone applies to is itself shaped by the discrimination you are measuring, then adjusting for department removes part of the very effect you are looking for. No amount of data resolves this; it takes an argument about how the world works.

Which is why the honest presentation is both. Report the stratum rates, the pooled rates and the standardised comparison together, and say what each answers. A single number here is a choice about the question, and presenting it alone hides that a choice was made.

Where it turns up outside the textbooks

Four settings where this is a live risk rather than a curiosity, and the strata that usually cause it.

Conversion rates by traffic source. A new channel can convert better on every device and worse overall, because it brings disproportionately more mobile traffic and mobile converts worse everywhere. Stratify by device before comparing channels.

Hospital and surgeon outcomes. The best surgeons take the hardest cases, so raw mortality rates rank them backwards. Every published surgical outcome table is risk-adjusted for exactly this reason, and unadjusted league tables actively punish the specialists.

Batting averages across seasons. A player can hit for a higher average than another in both of two seasons and a lower combined average, if their at-bat counts differ between a good year and a bad one. It has genuinely happened in Major League Baseball.

Pay gap analyses. An organisation can pay women more than men in every job grade and still report a large aggregate gap, because grade composition differs. Both numbers are real and they support opposite headlines — which is why the reporting standard asks for both.

The common signature is a comparison across groups that are not distributed alike. Whenever you are about to compare two rates, ask what the two populations differ in besides the thing you are measuring. If you can name it, stratify by it before you compare.

Related reversals that are not this one

Three effects get filed under this name and are different mechanisms with different fixes.

The ecological fallacy is about level, not reversal. A correlation between group averages need not hold within groups — and can even run the opposite way. Country-level income and some outcome can correlate strongly across 40 nations and be near zero inside each. Simpson’s paradox is one route to this, but the fallacy is the broader error of transferring a group-level finding to individuals.

Berkson’s paradox is created by selection. Two independent traits become negatively correlated among people selected for having at least one. Among hospital patients, two unrelated conditions appear to protect against each other; among admitted students, test scores and interview scores often correlate negatively. Stratifying does not fix this — the sample itself is the problem.

Regression to the mean is about repetition. An extreme measurement is followed by a less extreme one for reasons of noise alone, which makes any intervention applied to the extreme group look effective. No stratification helps; only a control group does.

What distinguishes Simpson’s paradox is that all the data is present and correct. Nothing was selected away, nothing was measured twice, and every rate in the table is accurate. The reversal comes purely from how the parts were weighted into the whole — which is why it can be repaired arithmetically, and the other three cannot.

Sources and methodology

References for both datasets and the adjustment.

• Bickel, Hammel & O’Connell, “Sex Bias in Graduate Admissions: Data from Berkeley”, Science 187, 1975 — the source of this page’s default dataset
• Charig et al., BMJ 292, 1986 — the kidney stone treatment comparison used as the second preset
• Simpson, “The Interpretation of Interaction in Contingency Tables”, JRSS B 13(2), 1951 — the paper the effect is named for
• Correlation Coefficient Calculator — including partial correlation, which faces the same limits as standardisation
• Probability Calculator — for the conditional probabilities the reversal is built from
• How we calculate — methodology

Method. Every stratum’s two rates and the two pooled rates are computed from the raw counts, so the reversal is detected rather than eyeballed. A reversal is reported only when a strict majority of the strata that have a direction contradict the pooled direction — a three-three split is not a reversal — and a total reversal is reported separately when every stratum disagrees with the total. Direct standardisation uses the combined stratum totals as weights, which is the conventional neutral choice, and the page says so because a different weight set answers a different question. The suite asserts the Berkeley figures to four decimals (44.5188% against 30.3542%, women higher in four of six departments), that standardising moves the gap from +14.16 to −4.26 percentage points and changes its sign, that the kidney stone data is a total reversal, and — checked by construction over generated data rather than by example — that exactly equal stratum rates make any reversal impossible while equal composition rules out only the total kind, since a bare majority can be outweighed by one large stratum. That engine is verified on every change against 40 assertions. The count and the per-case breakdown are published on the formula verification page.

Related calculators

Where this goes next:

More in Math, or browse all calculators.