X-bar and R Chart Calculator

Two sigmas, and their gap is the diagnosis

The chart prints two estimates of the process standard deviation. On a stable process they agree, and when they do not, the difference is the finding.

Within-subgroup sigma is the average range over d₂. It only ever sees variation inside a subgroup, so a shift between subgroups does not touch it.

Pooled sigma is the standard deviation of every observation. It contains the shift, the drift and anything else that moved.

On the step-change preset they are 2.1935 and 2.5830 — 17.8% apart. On the drifting preset, 1.8894 and 2.3436, a gap of 24.0%. On the stable preset, 2.0327 and 2.0976, under 3.2%.

Which is why the limits must come from the within estimate. Building them from the pooled figure widens the limits using the very variation they are supposed to detect, and a sufficiently bad process then draws limits wide enough to contain itself.

Where the constants come from

A2 = 0.577 at n = 5 is copied from a table in almost every treatment of this chart. It is a number with a derivation.

d₂ is the expected range of n standard normals — the integral of 1 − Φ(x)ⁿ − (1 − Φ(x))ⁿ over the whole line, evaluated here by Simpson’s rule.

A₂ is then just 3/(d₂√n). Three sigmas, divided by the √n that turns a process sigma into a sigma for the subgroup mean, divided by the d₂ that turns an average range into a sigma.

c₄ has a closed form and at n = 2 it is √(2/π), which says a two-point standard deviation runs 20.2% low on average.

Computing them earns two things. The chart works at subgroup sizes tables stop before, and the values can be checked against those tables rather than copied — which the verification suite does at twelve sizes for d₃ alone.

Eight rules, and what running them costs

The extra rules exist because a point outside the limits is a poor detector of anything except a large sudden shift. They are not free.

Rule 1 alone gives a false alarm about once every 370 points. That is the 3-sigma limit doing what it was designed to do.

Running all eight brings that to roughly one in 91. Four times as many false alarms, in exchange for catching shifts the limits cannot.

The step-change preset shows the trade paying off: ten subgroups flagged, only one of them outside the limits. The other nine are runs.

Choose the rules before looking at the chart. Applying rule 6 because a run happens to be visible is not a test, and a chart that is watched by people who investigate every alarm will be abandoned if four in five are noise.

The subgroup is the whole design

Everything on this page rests on what goes into one subgroup, and that is a sampling decision the arithmetic cannot check.

A subgroup should contain only common-cause variation. Five consecutive parts from one machine, taken within a minute — anything that could change between them belongs between subgroups instead.

Mixing two machines into one subgroup inflates the range, which widens the limits, which hides the very difference between the machines you wanted to see.

Spreading a subgroup over a shift does the same thing more slowly. The chart then reports that nothing is out of control, correctly and uselessly.

The tell is a chart that never signals and an R chart with a large average range. Compare the within-subgroup sigma against what you know the equipment can do; if it is far larger, the subgroups are too broad.

R or S, and why R won

The range chart is the default in most of industry and the standard deviation is the better statistic. The history explains the gap.

The range is one subtraction. In 1924 that mattered enormously, and it still matters on a shop floor where charts are filled in by hand.

The standard deviation uses every observation and the range uses two. At n = 5 the range is about 96% as efficient, so the loss is small; by n = 10 it is under 85%.

On the stable preset the two sigma estimates are 2.0327 from the range and 2.0384 from the standard deviation — 0.28% apart, which is what “good enough at n = 5” looks like.

The rule of thumb: R up to about n = 8, S above it. Both are on the toggle here, on the same data, so the difference can be looked at rather than argued about.

Control limits are not specification limits

This confusion is common enough to be worth stating on its own, because acting on it produces exactly the wrong behaviour.

Control limits describe what the process does. They come entirely from the data and know nothing about what anyone wants.

Specification limits describe what the customer will accept. They come from a drawing and know nothing about the process.

A process can be in perfect control and produce nothing acceptable, and a process can meet every specification while being wildly unstable.

Drawing specification limits on a control chart is the error to avoid. It invites adjusting the process every time a point drifts toward a tolerance, which adds variation rather than removing it. The question of whether the process meets the specification is a capability question, and it only means anything once the chart says the process is stable.

Reporting a control chart

Four items, and the third is the one that makes the limits reproducible.

Give the subgroup size and how subgroups were formed. The whole analysis depends on it and no reader can reconstruct it.

Give the number of subgroups the limits were computed from. Limits from fewer than about twenty subgroups are unstable, and from fewer than ten are little more than a guess.

Say which rules you applied. “Out of control” means something different under one rule than under eight.

And say whether the limits were recomputed. Removing an assignable-cause subgroup and refitting is standard practice and has to be stated, because the limits after the removal are not the limits before it.

Sources and methodology

References for the charts and their constants.

• Shewhart, “Quality Control Charts”, Bell System Technical Journal 5(4), 1926
• Nelson, “The Shewhart Control Chart — Tests for Special Causes”, Journal of Quality Technology 16(4), 1984
• Montgomery, “Introduction to Statistical Quality Control”, 8th edition, Wiley, 2019 — the constant tables this reproduces
• Individuals Chart Calculator — the same idea when subgroups are impossible
• Cpk Calculator — whether a stable process meets its specification
• How we calculate — methodology

Method. Every Shewhart constant is computed from its definition rather than taken from a table. d₂ comes from Simpson’s rule on the integral of 1 − Φ(x)ⁿ − (1 − Φ(x))ⁿ; d₃ from a two-dimensional Simpson rule over the joint density of the smallest and largest order statistics, which reaches more precision than any published table carries and, unlike a simulation, returns the same value every time; c₄ in closed form. The suite checks all of them against the published values — d₂ at seven subgroup sizes, d₃ at twelve, c₄ at seven, and A₂, A₃, B₃, B₄, D₃ and D₄ too — and checks the closed forms where they exist: d₂(2) is 2/√π, d₃(2) is √(2 − 4/π) and c₄(2) is √(2/π). It also asserts the cross-engine identity that ties c₄ to the Hedges correction in the effect-size engine: c₄(n) = J(n)·√(n/(n−1)), to machine precision at nine sizes. Sigma for the limits always comes from within-subgroup variation, never from the pooled standard deviation, and both are printed so the gap is visible. That engine is verified on every change against 139 assertions. The count and the per-case breakdown are published on the formula verification page.

Related calculators

Where this goes next:

More in Math, or browse all calculators.