Two rates compared, with an interval for the difference between them.
Two rates, one pooled standard error
A 4.92% conversion against 4.00%. A plausible-looking lift, and p = 0.124 — the sample does not support it.
A 4.9167% (118/2400) · B 4.0000% (96/2400)
z = 1.53858, p = 0.123907
The difference is -0.9167 percentage points, with a 95.000% interval of -0.255 to 2.096pp. The interval includes zero, so a difference of nothing is among the values this data is consistent with.
z statistic
1.53858
pooled standard error
p-value
0.123907
two-sided
Difference
-0.9167pp
A − B
Newcombe interval
-0.255 to 2.096
percentage points
Relative risk
1.22917
A ÷ B
Odds ratio
1.24102
further from 1
Chi-square
2.36722
= z², the same test
Wald interval
-2.084 to 0.251
the textbook version
Under the null the two groups share one underlying rate, which is why the standard error is built from the POOLED proportion of 4.4583% rather than from the two rates separately. Using the separate rates is correct for the confidence interval and wrong for the test.
z² = 2.367221 and the chi-square statistic of the same 2×2 table is 2.367221. Identical, to the last digit the arithmetic allows. The two-proportion z-test and the chi-square test of independence on a 2×2 table are the same procedure written two ways, so choosing between them changes nothing — the only real choice is whether to apply a continuity correction, which this tool does not. The identity is verified across 400 random tables in the verification suite.
Newcombe gives -0.255 to 2.096pp; the Wald interval gives -2.084 to 0.251pp. The Wald interval for a difference inherits every problem the single-proportion Wald interval has: it can run past ±1 and it undercovers at small counts. Newcombe’s method builds the difference interval out of two Wilson intervals instead, which fixes both.
A p-value is not an effect size, and 200 people is not 2,000. A difference of 0.9167 percentage points can be highly significant on a large sample and invisible on a small one, without the underlying difference changing at all. The interval above is the part that says how large the effect might be; the sample size calculator says how many you would need to detect it reliably.
What this tool shows
z² equals the chi-square statistic of the same 2×2 table, exactly. The two-proportion z-test and the chi-square test of independence are one procedure written two ways, verified to better than 1e-8 across 400 random tables. The tool computes both, plus a Newcombe confidence interval for the difference and a flag when the counts are too small for either.
The pooled two-proportion z-test and its two-sided p-value
A Newcombe confidence interval for the difference, built from two Wilson intervals
The Wald interval beside it, so the gap at small counts is visible
The equivalent chi-square statistic, computed independently
Relative risk and odds ratio from the same table
A warning, and a route to Fisher’s exact test, when a cell is under 10
Pooled z-test Newcombe interval z² = chi-square Small cells flagged
The test pools; the interval does not. Both are right.
Updated 12 September 2026 · Works in any browser, no installation
The test asks whether two observed rates differ by more than sampling variation would explain. Under the null they come from one underlying rate, so the standard error is built from the POOLED proportion — and the p-value is the area in both tails beyond the resulting z. The interval for the difference answers the separate question of how large that difference might be.
At a glance
Formula shown
z = (p̂₁ − p̂₂) / √(p̄(1 − p̄)(1/n₁ + 1/n₂)), where p̄ = (x₁ + x₂)/(n₁ + n₂) is the pooled proportion. Pooling is correct for the TEST because the null says both groups share one rate. The confidence INTERVAL uses the two rates separately, because under the alternative they do not — Newcombe’s method combines two Wilson intervals rather than adding two Wald standard errors. z² equals the Pearson chi-square statistic of the same 2×2 table without continuity correction.
Scenario support
A/B tests and conversion-rate comparisons, treatment versus control event rates, defect rates between two lines or suppliers, response rates between two segments, and any comparison of “x of n” against “y of m”.
Educational estimate
Planning support from the values you enter — not professional advice.
Why the test pools and the interval does not
The tool uses one standard error for the p-value and a different construction for the interval. That looks inconsistent and is not.
The test assumes the null is true while computing. Under the null both groups share one underlying rate, so the best estimate of it uses all the data: the pooled proportion (x₁ + x₂)/(n₁ + n₂). Estimating two separate rates there would be estimating a difference the hypothesis says does not exist.
The interval makes no such assumption. It has to cover the true difference whatever it is, so it uses each group’s own rate. Pooling there would assume the answer.
Which is why a p-value and an interval can sit fractionally either side of the threshold. They are built from different standard errors, deliberately, and on borderline data that shows.
Software that uses one standard error for both hides the distinction and gets one of the two slightly wrong. The tool prints both so the difference is visible rather than reconciled by force.
The chi-square test is the same test
“Should I use a z-test or a chi-square test for my 2×2 table?” is one of the most asked questions in applied statistics, and it has no content.
z² IS the Pearson chi-square statistic. Not approximately — algebraically identical, for any 2×2 table. The suite verifies it across 400 random tables with a worst discrepancy below 1e-8, which is floating-point noise rather than a difference.
A chi-square with 1 degree of freedom is the square of a standard normal, so the p-values match as well. The tool prints both statistics so the identity is checkable on your own numbers.
The only real difference is the continuity correction. Some chi-square implementations apply Yates’s correction by default and most z-tests do not, which produces a genuine discrepancy — and it is a difference of convention, not of test. This tool applies none.
The z-test has two practical advantages. It has a sign, so it says which group is higher, and it extends directly to a one-sided test. Chi-square is always two-sided because squaring discards the direction.
Chi-square generalises where z does not: to tables larger than 2×2, where there is no single difference to take. For two groups and two outcomes, use whichever you find easier to explain.
Newcombe’s interval, and why not Wald
The textbook interval for a difference of proportions adds two Wald standard errors. It inherits every problem the single-proportion Wald interval has, and adds one of its own.
It can run past ±1. A difference of proportions lives in [−1, 1], and the Wald interval does not know that. On small counts near a boundary it regularly produces bounds outside the possible range.
It undercovers, for the same reason the single-proportion version does. Both components assume the estimate is normal with a variance estimated from itself, and both assumptions fail where the counts are small.
Newcombe’s method builds the difference interval from two Wilson intervals instead: it takes how far each proportion’s own interval extends and combines those distances in quadrature. The result cannot leave [−1, 1] and holds its coverage far better at small n.
The tool prints both so the gap is visible where it matters — and invisible where it does not, since on large balanced samples the two agree closely.
The interval is the more useful output. “p = 0.011” says the difference is probably not zero; “−26.1 to −3.4 percentage points” says how large it might be, which is what a decision needs.
When the counts are too small
Every figure in this test rests on a normal approximation to a discrete distribution, and that approximation has a floor.
The usual condition is at least 5 expected in every cell, or 10 in stricter treatments. The tool checks the observed cells and flags when the smallest falls below 10.
Below that, Fisher’s exact test is the right tool. It enumerates the hypergeometric distribution instead of approximating it, needs no minimum cell count, and is exact by construction.
Fisher’s test is conservative, which is a real cost. Its discrete p-values mean the realised error rate sits below the nominal level, so it has less power than the approximation it replaces. Exactness is not free.
And there is more than one two-sided convention for it, which is why the Fisher page computes all three — they give different p-values on the same table.
A zero cell is the extreme case. With no events in one arm the odds ratio is undefined and the pooled standard error still computes happily, which is exactly the situation where an exact method is not optional.
A/B tests: what this does and does not settle
Two conversion rates is the most common use of this test and the one with the most ways to go wrong outside the arithmetic.
Peeking invalidates the p-value. Running the test repeatedly as data accumulates and stopping when it crosses 0.05 will cross 0.05 eventually with probability approaching 1, whatever the truth. A fixed sample size decided in advance, or a sequential method built for it, is the fix.
Multiple variants need a correction. Testing four variants against a control is four comparisons, and the chance of at least one false positive is far above 5% — see the Bonferroni calculator for what that costs and what the alternatives are.
A significant result is not a big result. The A/B preset here shows 4.92% against 4.00% on 2,400 per arm: a plausible-looking lift that the data does not support (p = 0.124). Doubling the sample would not change the lift, only whether it is detectable.
And the interval is what you should act on. If the whole interval sits above the difference you would need to justify shipping, ship. If it spans that threshold, the experiment has not answered the question regardless of the p-value.
Reporting a comparison of two rates
Four things, and the first two are usually the ones omitted.
Report both rates with their counts. “4.92% vs 4.00% (118/2400 vs 96/2400)” is complete. “A 23% lift” is not, and cannot be made complete by the reader.
Report the difference with an interval. In percentage points, not as a ratio. A ratio without its baseline has the same problem the relative risk page describes at length.
Say whether the sample size was fixed in advance. A test stopped when it became significant is a different procedure from one run to a planned n, and the p-value means something different in each.
Say how many comparisons were made. One test out of one and one test out of twelve produce the same p-value and very different evidence.
Method. The z-statistic uses the pooled proportion in its standard error, which is correct under the null, while the confidence interval uses Newcombe’s method built from two Wilson intervals — a deliberate difference rather than an inconsistency, and both are printed. The identity between this test and the 2×2 chi-square is verified rather than asserted: z squared matches the independently computed Pearson statistic to better than 1e-8 across 400 randomly generated tables. The Wald interval for the difference is computed alongside so the divergence at small counts is visible, and the tool routes to an exact method when the smallest cell falls below 10. That engine is verified on every change against 75 assertions. The count and the per-case breakdown are published on the formula verification page.
Related calculators
Where this goes next:
One-Proportion Z-TestThe score z-test with the exact binomial test beside it: 60 of 100 against 0.5 gives p = 0.0455 by one and 0.0569 by the other — opposite verdicts at the conventional threshold, on identical data.
Wilson Score IntervalWilson, Wald, Agresti-Coull and Clopper-Pearson on one set of counts, with the EXACT coverage each delivers at your sample size — a "95%" Wald interval covers the truth 80.85% of the time at n = 30, p = 0.1.
Fisher's Exact TestThe exact p for a 2×2 table under all three two-sided conventions, because they disagree — 0.0406 against 0.0699 on the built-in table, across the 5% line — plus the test's actual size by enumeration, which is 2.30% at a nominal 5%.
Chi-SquareGoodness of fit and tests of independence with every expected count and per-cell contribution shown — because the validity condition is about expected counts, not observed ones, and most calculators hide them.
Relative RiskRisk ratio and odds ratio from one table with the divergence between them plotted: they agree to half a percent at a 1% baseline, and at an 80% baseline the odds ratio is exactly half the risk ratio.
Sample SizeResponses needed for a target margin of error, with the finite-population correction and a table of the whole cost curve — because n scales with 1/margin², so the last point of precision costs more than the first ten.
An educational tool. The test assumes two independent random samples — it cannot detect that the groups were assigned non-randomly, that the same person appears in both, or that the test was run repeatedly until it produced this result.
Published the pooled two-proportion z-test with a Newcombe confidence interval for the difference, and verified the identity that settles the recurring question: z squared equals the Pearson chi-square statistic of the same 2x2 table to better than 1e-8 across 400 randomly generated tables.
The test pools and the interval does not, deliberately - the null says both groups share one rate, so pooling is correct there, and the interval has to cover the true difference whatever it is. Both are printed rather than reconciled by force.
Newcombe's method is used instead of the Wald difference for the same reason Wilson beats Wald on a single proportion, and the Wald interval is shown alongside so the divergence at small counts is visible.
The A/B preset ships an honest negative: 118/2400 against 96/2400 is a 4.92% rate against 4.00%, and p = 0.124 - a plausible-looking lift the sample does not support.
A smallest cell below 10 triggers a route to Fisher's exact test rather than a silent approximation.
Add this calculator to your site
Responsive embed — and private: nothing your visitors type leaves their browser.