Five corrections at once, and the one that makes Bonferroni obsolete.
Five corrections, on one set of p-values
Bonferroni keeps one and Holm keeps three — identical error control, triple the findings.
5 tests at α = 0.050, Holm
3 of 5 survive
Uncorrected, 5 tests at 5.0% carry a 22.62% chance of at least one false positive. This method controls the family-wise error rate.
Bonferroni
1 of 5
controls FWER
Holm
3 of 5
controls FWER
Šidák
1 of 5
controls FWER
Holm-Šidák
3 of 5
controls FWER
Benjamini-Hochberg
3 of 5
controls FDR
Each raw p-value with its adjusted value under all five correction methods
Raw p
Bonferroni
Holm
Šidák
Holm-Šidák
Benjamini-Hochberg
0.00400
0.02000 ✓
0.02000 ✓
0.01984 ✓
0.01984 ✓
0.02000 ✓
0.01200
0.06000
0.04800 ✓
0.05858
0.04714 ✓
0.02667 ✓
0.01600
0.08000
0.04800 ✓
0.07748
0.04724 ✓
0.02667 ✓
0.30000
1.00000
0.60000
0.83193
0.51000
0.37500
0.50000
1.00000
0.60000
0.96875
0.51000
0.50000
Holm keeps 3 and Bonferroni keeps 1, on identical data and with identical error control. Holm is a step-down procedure: it compares the smallest p against α/m, the next against α/(m−1), and so on, stopping at the first failure. It controls the family-wise error rate exactly as Bonferroni does, and it is uniformly more powerful — there is no set of p-values on which Bonferroni rejects something Holm does not. The verification suite checks that across 2,000 random families and finds zero violations. Plain Bonferroni is dominated, and has been since Holm published in 1979.
Šidák is exact under independence and Bonferroni is a bound, which is why Šidák is always slightly less strict. 1 − (1 − α)^m is the exact family-wise rate when the tests are independent; m·α is the union bound on it, and the bound is always at least as large. With m = 20 at α = 0.05 the exact threshold is 0.002561 and Bonferroni’s is 0.0025 — a small difference, and free. The catch is the independence: correlated tests make Šidák anti-conservative, while Bonferroni holds regardless of dependence, which is the one real argument for it.
m is the number of tests you ran, not the number you are reporting. That is where correction most often goes wrong. Analysing twelve outcomes and correcting the three that looked promising controls nothing — the selection already happened. The same applies to subgroups explored and dropped, to models tried and discarded, and to an analysis rerun after adding data. A correction can only account for comparisons it is told about.
What this tool shows
Holm controls exactly what Bonferroni controls, and never rejects fewer. Across 2,000 random families of p-values the verification suite finds zero exceptions. On the preset here Bonferroni keeps one result and Holm keeps three. Plain Bonferroni is dominated, and has been since 1979.
Bonferroni, Holm, Šidák, Holm-Šidák and Benjamini-Hochberg on the same p-values
Adjusted p-values per test, so each can be compared against your own alpha
Which quantity each method controls — family-wise error, or false discovery rate
The uncorrected family-wise error rate for your number of tests
How many results each method keeps, side by side
Why m is the number of tests run, not the number reported
Five methods Holm dominates FDR is different Adjusted p per test
Same error control, three findings instead of one.
Updated 12 September 2026 · Works in any browser, no installation
Running many tests at 5% each does not give you a 5% error rate. Twenty independent tests carry a 64% chance of at least one false positive. A correction reduces the per-test threshold so the rate across the whole family holds — and the methods differ in how much power that costs, and in what exactly they hold.
At a glance
Formula shown
Bonferroni compares every p against α/m, or equivalently multiplies each p by m. Šidák uses the exact independent rate: 1 − (1 − p)^m. Holm steps down the sorted p-values, comparing the smallest against α/m, the next against α/(m − 1), and stopping at the first failure. Benjamini-Hochberg steps up from the largest, comparing p₍ₖ₎ against (k/m)·α, and controls the expected proportion of false positives among rejections rather than the chance of any.
Scenario support
Multiple outcomes in a trial, several subgroups, many genes or markers, a set of planned contrasts after an ANOVA, repeated A/B tests, or any analysis where more than one hypothesis was tested and only some are being reported.
Educational estimate
Planning support from the values you enter — not professional advice.
Holm dominates Bonferroni, and nothing about that is close
This is the single most useful thing on the page, and it is not a matter of preference or of trading power against rigour.
Holm controls the family-wise error rate exactly as Bonferroni does. Same guarantee, same assumption-free validity under any dependence structure. Nothing is given up.
And it never rejects fewer hypotheses. Not usually — never. Its adjusted p-values are never larger than Bonferroni’s for any test on any data. The verification suite checks this across 2,000 random families and finds zero violations.
The mechanism is a step-down. Sort the p-values. Compare the smallest against α/m. If it clears, compare the next against α/(m − 1) — because one hypothesis has been resolved and only m − 1 remain. Continue until one fails, then stop.
On the preset that turns one finding into three. p = 0.004, 0.012 and 0.016 against Bonferroni’s flat 0.01 threshold gives one survivor; against Holm’s 0.01, 0.0125 and 0.01667 it gives three.
The improvement is not common on random p-values — about 2% of families. It shows up precisely when several tests sit near the threshold, which is the situation where the decision matters. And since it costs nothing, the rate is beside the point: there is no dataset on which choosing Bonferroni is better.
Benjamini-Hochberg answers a different question
It is usually presented as a more permissive correction, which makes it sound like a weaker version of the same thing. It is not on the same scale at all.
A family-wise method holds the probability of ANY false positive at α. One false positive in the whole set is a failure.
Benjamini-Hochberg holds the expected PROPORTION of false positives among the results you keep. Reject twenty and about one of them being wrong is not a failure — it is the specification.
The borderline preset makes the gap vivid. Five p-values between 0.04 and 0.049: every family-wise method keeps none, and Benjamini-Hochberg keeps all five. Neither answer is wrong; they are answers to different questions.
Which one you want follows from what happens next. A screen that generates candidates for follow-up can absorb a known share of false leads, and controlling the family-wise rate there throws away real findings for a guarantee nobody needed. A confirmatory test where one false positive changes practice cannot.
The tool labels which quantity each method controls, rather than listing five numbers and letting the smallest look like the most permissive. That label is the part that decides which column to read.
Šidák is exact and Bonferroni is a bound
The two are close enough that the difference rarely changes a decision, and the reason they differ is worth knowing because it is the one real argument for Bonferroni.
Under independence the exact family-wise rate is 1 − (1 − α)^m. Šidák inverts that, so its threshold is exactly right when the tests are independent.
Bonferroni’s m·α is the union bound on that rate, and a bound is always at least as large as what it bounds. So Šidák’s threshold is always slightly looser, and Bonferroni is always slightly conservative.
The gap is small. At m = 20 and α = 0.05 the exact threshold is 0.002561 and Bonferroni’s is 0.0025. It grows with m but stays modest.
The catch is the independence. Correlated tests make Šidák anti-conservative — it can exceed its nominal rate. Bonferroni holds under any dependence structure, which is why it survives in settings where the tests are obviously related.
And Holm inherits that robustness. Holm is valid under arbitrary dependence, like Bonferroni, while being more powerful. Holm-Šidák is more powerful still and carries Šidák’s independence assumption with it — so the safe default is Holm, and the aggressive one is Holm-Šidák if you can defend independence.
m is what you tested, not what you are reporting
Every correction takes one input that is not a p-value, and it is the input most often wrong.
m is the number of hypotheses tested. Not the number that looked interesting, not the number that made the paper, and not the number of rows in the table.
Correcting the subset you chose to report controls nothing. The selection already happened. Testing twelve outcomes, picking the three with the smallest p-values, and correcting for three gives a procedure with no error guarantee at all — it is arithmetic applied after the fact.
The same applies to everything that was tried. Subgroups explored and dropped, models fitted and discarded, outcomes measured and not mentioned, an analysis rerun after collecting more data. Each is a comparison, and a correction can only account for comparisons it is told about.
Which is why pre-registration and corrections are the same argument. Writing down the hypotheses before seeing the data fixes m at a number that can be defended. Without that, m is whatever the analyst remembers.
And it argues for correcting over a smaller, planned family. Three pre-specified contrasts corrected for three is far more powerful than three chosen from twelve corrected for twelve — and honest, which the latter is not. That is also why Tukey’s HSD corrects for all pairs: it assumes you will look at all of them, which after an ANOVA you will.
What a correction actually costs
Corrections are sometimes resisted as a tax on discovery. The cost is real, and it is specific rather than general.
They only ever cost you the marginal results. A p-value of 0.0001 survives twenty comparisons; one of 0.04 does not survive two. The winner preset shows it — corrections remove near-misses and leave clear findings untouched.
The power loss is a power loss, and it is quantifiable. Testing at α/m instead of α reduces the chance of detecting a real effect, and the right response is a larger sample rather than a looser threshold. The power calculator takes alpha as an input for exactly this reason.
Uncorrected multiplicity is not a smaller problem than lost power. Twenty tests at 5% carry a 64% chance of at least one false positive. That is not a subtle bias; it is a coin flip weighted toward finding something that is not there.
Fewer, better-chosen tests beat corrections. One pre-specified primary outcome needs no correction at all. Everything else can be reported as exploratory — which is honest, costs nothing, and is what the distinction exists for.
And an effect size with an interval survives all of this. Adjusted p-values answer a yes-or-no question about a family; a confidence interval on each effect says what was estimated and how precisely, and does not become meaningless because five other things were measured.
Reading an adjusted p-value
The output is designed so that adjusted p-values can be compared against your original alpha directly, and that convention has two edges worth knowing.
An adjusted p is not a probability of anything. Bonferroni’s is a raw p-value multiplied by m, capped at 1. It is a device for keeping the comparison threshold at α rather than a rescaled measure of evidence.
Which is why they cap at 1 and why several can be identical. Step-down methods force the adjusted values to be monotone in the raw ones, so once a step fails every later one is held at least as high — producing runs of the same number, as the table shows.
The step-down structure means results are not independent of each other. Under Holm, whether your fourth test survives depends on the first three. Removing one test from the family can change the verdict on another, which is a real property rather than a bug.
Report the raw p-values as well. They are what a reader needs to recompute anything, to apply a different correction, or to include your results in a meta-analysis. An adjusted value alone is not recoverable without knowing m and the method.
And name the method. “Corrected for multiple comparisons” describes five procedures that give different answers on the same data, two of which control different quantities. The sentence is not informative without the word after it.
Sources and methodology
References for the procedures and what each one controls.
Method. All five methods are computed on the same set, with step-down and step-up procedures enforcing monotonicity so the adjusted p-values cannot invert the ordering of the raw ones. The suite asserts the central claim as a property rather than an example: across 2,000 random families Holm never rejects fewer than Bonferroni and its adjusted p-values are never larger, while rejecting strictly more on about 2% of them. It also checks that Šidák is never more conservative than Bonferroni across 1,000 families, that Benjamini-Hochberg never rejects fewer than Holm, that adjusted p-values are monotone under all five methods, and that the result labels which quantity each controls. That engine is verified on every change against 47 assertions. The count and the per-case breakdown are published on the formula verification page.
Related calculators
Where this goes next:
Tukey HSDEvery pairwise comparison after an ANOVA with simultaneous confidence intervals — and the studentized range integrated rather than interpolated from a table, so any group count, df and level works. At two groups the critical q is exactly √2 times the critical t.
Statistical PowerPower and sample size from the non-central t rather than a normal approximation, with the gap shown — plus a live demonstration that post-hoc power is a function of the p-value alone, and 0.500044 at p = 0.05 for every study ever run.
p-valueA p-value from a t or z statistic, one- or two-tailed — with a panel that holds an effect fixed and grows the sample, so you can watch significance appear from nothing but n.
One-Way ANOVAThe full F table with eta and omega squared, plus every pairwise gap — because a significant F says something differs and never says which, and ten groups tested pairwise carry a 90% false-positive rate.
Confidence IntervalIntervals for a mean or a proportion using t at every sample size and Wilson rather than the textbook Wald formula — with both methods shown, because Wald returns [0,0] at zero successes.
Kruskal-WallisApplies the tie correction and shows it against the uncorrected value, because on ordinal data it moves p from 0.054 to 0.027 — across the conventional threshold, on identical data.
An educational tool. A correction only accounts for the comparisons it is told about; applying one to a subset selected after seeing the results provides no error control. Family-wise methods and the false discovery rate control different quantities and are not interchangeable.
Published a multiple-comparison calculator built around a fact rather than a menu: HOLM DOMINATES BONFERRONI. It controls exactly the same quantity, the family-wise error rate, is valid under the same arbitrary dependence, and never rejects fewer hypotheses. The suite checks that across 2,000 random families with zero violations. On the built-in preset Bonferroni keeps one result and Holm keeps three.
Labels which quantity each method controls, because Benjamini-Hochberg is not a weaker family-wise correction — it controls the false discovery rate, the expected share of false positives among the rejections. On the borderline preset every family-wise method keeps none and Benjamini-Hochberg keeps all five, which is not a disagreement about strictness but about the question.
Computes all five methods on the same set with monotonicity enforced on the step-down and step-up procedures, so adjusted p-values cannot invert the ordering of the raw ones. Šidák is shown to be never more conservative than Bonferroni, with the independence assumption that buys it stated.
Says plainly that m is the number of tests RUN, not the number being reported. Correcting the subset that looked promising controls nothing, because the selection already happened — which is the most common way a correction is applied and still provides no guarantee.
Add this calculator to your site
Responsive embed — and private: nothing your visitors type leaves their browser.