Normality Test Calculator

The window where a normality test is informative is narrow

The test is an honest answer to a question almost nobody is asking. It answers “can I reject normality”; people want to know “is my data normal enough”.

At small n, no test has the power to detect a departure that matters. A lognormal with a skewness of 0.72 — visibly bent on a histogram — is caught by Shapiro-Wilk 9.6% of the time at n = 10. You fail to reject and learn nothing, because failing to reject was the overwhelmingly likely outcome whatever the truth was.

At large n, every test rejects. The same lognormal is caught 100% of the time by n = 1,000. So is a departure a tenth the size, and so is the rounding in your measuring instrument. You reject and learn nothing, because rejecting was inevitable.

Both ends were measured rather than reasoned about. The sweep in the verification suite runs Shapiro-Wilk 600 times at each of six sample sizes against that lognormal, and the rejection rate climbs monotonically from 9.6% to 100%.

Which is why the tool resamples your own data. The table under the Q-Q plot draws new samples, with replacement, from the values you entered, and reports how often Shapiro-Wilk would reject at each size. If the rate at your n is 15%, a non-significant result was the likely outcome regardless. If it is 99%, your n had plenty of power and the result means something.

None of this makes the test useless. It makes the p-value one input rather than a verdict, and it makes the Q-Q plot the thing to read first.

Why three tests, and what each one cannot see

They are not three attempts at the same measurement. They look at different features and they fail in different places.

Shapiro-Wilk asks how straight the Q-Q plot is. W is the squared correlation between your sorted values and the positions a normal would have put them. It is the most powerful general-purpose test against most alternatives, which is why it is the default here.

Anderson-Darling weights the tails. Its integral gives the extremes more influence than the middle, so it catches heavy-tailed departures that a whole-shape test can average away. When it rejects and Shapiro-Wilk does not, look at the ends of the Q-Q plot.

Jarque-Bera reads two numbers and nothing else — skewness and excess kurtosis. Anything that leaves both near their normal values is invisible to it. A symmetric bimodal mixture, a uniform, a gap in the middle of the range: all pass.

That blind spot was measured. On uniform samples of 60, Shapiro-Wilk rejects 85.8% of the time, Anderson-Darling 69.0%, and Jarque-Bera 0.6%. A flat distribution is about as non-normal as a symmetric shape can be, and choosing Jarque-Bera for it is choosing not to find out.

So disagreement between the three is a signal, not a problem. All three rejecting means a gross departure. Shapiro-Wilk alone rejecting points at shape. Anderson-Darling alone points at tails. Jarque-Bera alone — rare — usually means one extreme outlier moving the moments.

Verified by calibration, not by a textbook number

There is no reference implementation to diff these against, so they are checked the way a test is actually supposed to behave.

A correct test at α = 0.05 rejects a TRUE null 5% of the time. Two thousand genuinely normal samples were generated at each of five sample sizes, and the realised rejection rates were 4.53%, 5.17%, 5.45%, 4.97% and 4.92% for Shapiro-Wilk at n = 10, 20, 30, 50 and 100.

That checks the statistic AND the p-value transform together. A wrong W would move the rate. A wrong Royston transform would move the rate. Copying a golden value out of a textbook checks neither, because a single number can be right by coincidence in a broken implementation.

Anderson-Darling came out at 4.42% to 5.42% across the same range. Both tests are calibrated where they claim to be.

Jarque-Bera is not, and the same run found it. Its asymptotic chi-square approximation gives a realised rate of 0.55% at n = 10, 2.55% at n = 20, and only 4.38% by n = 100. At small n it is far more conservative than advertised — a real property of the test, not a defect here, and one worth knowing before reading its p-value as if it meant 0.05.

Invariances are checked too. All three statistics must be unchanged by shifting the data, rescaling it, or reordering it, since none of those change the shape. Each is verified rather than assumed.

Read the Q-Q plot before the p-value

The tests compress a whole distribution into one number. The plot shows which part of it disagrees, and that is the part that determines whether you need to do anything.

Points bending upward at the right is right skew. The largest observations are further out than a normal would put them. This is the commonest departure in real data and the one most often fixed by a log transform.

An S-shape is a kurtosis problem. Both ends deviating in opposite directions means the tails are heavier or lighter than normal while the middle is fine.

One or two points far off an otherwise straight line are outliers, not a shape. That is a different problem with a different fix, and the outlier calculator is the tool for it. Deleting them changes the answer; deciding whether to is not a statistical question.

Steps or flat runs mean discreteness. Data rounded to whole units, or a scale with few possible values, will fail a normality test at large n no matter how symmetric it is. That failure is real and usually harmless.

The reference line here is mean + SD·z rather than a fitted line through the points, so it shows where the data would be if the normal with YOUR mean and SD were correct — not where the best-fitting normal would put it.

When normality matters, and when it does not

The reason to test is usually an assumption in another procedure, and those assumptions vary enormously in how much they care.

The t-test does not assume your data is normal. It assumes the sampling distribution of the mean is, which the central limit theorem delivers for moderate n from almost any shape. Testing the raw data and then abandoning a t-test at n = 200 is a common and unnecessary move.

Regression assumes it of the residuals, not the variables. A skewed predictor is irrelevant; skewed residuals matter, and mainly for the prediction intervals rather than the coefficients.

Process capability assumes it absolutely. A Cpk of 1.33 corresponds to 33.04 defects per million only under a normal distribution. On a skewed process that figure can be wrong by orders of magnitude, and this is the case where a normality test earns its place.

Prediction intervals and tolerance intervals assume it in the tails — exactly where samples carry least information. A test that passes at n = 30 says almost nothing about the behaviour at the 99th percentile.

And where it genuinely fails, the alternatives are not a downgrade. The Wilcoxon signed-rank test and the Mann-Whitney U test give up very little power on normal data and a great deal less on skewed data.

Reporting a normality check

Four habits that make the result mean something.

Name the test. “Normality was confirmed (p > 0.05)” does not say which test, and on the uniform preset here that sentence is true for one test and false for another.

Never write “confirmed”. A non-significant result is a failure to reject. The null hypothesis of normality cannot be accepted by a test designed to reject it, and at small n it will not be rejected whatever the truth.

Report the sample size beside the p-value. They are not independent pieces of information — the p-value is close to uninterpretable without the n, in both directions.

Say what you did next. A normality test that changes nothing about the analysis did not need running. A normality test that changed the analysis should have that decision written down, including the case where it was rejected and you proceeded anyway because the sample was large enough for the central limit theorem to cover you.

And do not test after choosing the test. Running a normality check, then picking parametric or non-parametric on the result, then reporting the second p-value as if it were the only one computed is a multiple-comparison problem with no standard correction. It is common, and it inflates the error rate of the analysis it was meant to protect.

Sources and methodology

References for the three tests and their calibration.

• Shapiro & Wilk, “An Analysis of Variance Test for Normality (Complete Samples)”, Biometrika 52, 1965
• Royston, “Algorithm AS R94: Shapiro-Wilk W Test”, Applied Statistics 44(4), 1995 — the approximation used here
• Anderson & Darling, “A Test of Goodness of Fit”, JASA 49(268), 1954
• Jarque & Bera, “Efficient Tests for Normality, Homoscedasticity and Serial Independence”, Economics Letters 6(3), 1980
• Skewness Calculator — one of the two numbers Jarque-Bera reads
• Central Limit Theorem Calculator — why the raw data often does not need to be normal
• How we calculate — methodology

Method. Shapiro-Wilk follows Royston’s AS R94: Blom’s approximation to the expected normal order statistics, with polynomial corrections to the two extreme coefficients, and the log-normal transform of W for the p-value. Anderson-Darling uses the small-sample adjustment for estimated parameters. Because no reference implementation is available to diff against, the suite validates by CALIBRATION instead: two thousand samples from a true normal at each of five sample sizes, checking that each test rejects near 5% of the time — which tests the statistic and the p-value transform together. The same approach measured Jarque-Bera’s small-sample conservatism (0.55% at n = 10) and the power sweep against a mildly skewed alternative. Location, scale and order invariance are checked for all three. That engine is verified on every change against 44 assertions. The count and the per-case breakdown are published on the formula verification page.

Related calculators

Where this goes next:

More in Math, or browse all calculators.