Sensitivity and Specificity Calculator

Why a 99% test can be wrong half the time

Take a test that is 99% sensitive and 99% specific — better than most tests in routine use. Apply it to 10,000 people where 1% have the condition.

100 people are sick; the test finds 99 of them. 9,900 people are well; the test wrongly flags 1% of them, which is 99 people.

So there are 198 positive results and 99 of them are wrong. The positive predictive value is exactly 50%. A positive result from an excellent test is a coin flip, and nothing is broken.

The arithmetic is just the group sizes. A 1% error rate applied to 9,900 well people produces about as many false positives as a 99% detection rate applied to 100 sick ones. When the well outnumber the sick a hundred to one, a small false-positive rate is not small in absolute terms.

Drop the prevalence to 0.1% and the PPV falls to 9.02%. Nine in ten positives are now wrong. Raise it to 50% and the PPV is 99%. Same test, same instrument, same cut-off.

This is why screening an unselected population is a different problem from confirming a suspicion. A test that performs well in a clinic where a third of patients have the condition can be useless as a mass screen, and the validation study will not say so — because it was run at the clinic’s prevalence.

Four numbers, two of which move

The tool prints sensitivity, specificity, PPV and NPV together, then lets you change the prevalence and watch which ones move. Only two do.

Sensitivity and specificity are measured DOWN the columns of the 2×2 table — among the sick, and among the well. Those groups are defined by the true status, which the test cannot influence, so the proportions are properties of the instrument.

Predictive values are measured ACROSS the rows — among those who tested positive, and among those who tested negative. Those groups are mixtures, and the mixture depends entirely on how many sick and well people were fed in.

That is the whole distinction, and it is the same reversal as P(A|B) against P(B|A). Sensitivity is P(positive | sick); PPV is P(sick | positive). Bayes’ theorem converts one into the other, and prevalence is the term it needs.

So a study reporting only sensitivity and specificity has told you about the test and left you to work out what a result means in your setting. That is the correct division of labour, and it is routinely read as if the study had answered the second question too.

Likelihood ratios are the numbers that travel

If predictive values do not transfer between populations and sensitivity alone does not answer the clinical question, something has to bridge them. Likelihood ratios do.

LR+ = sensitivity ÷ (1 − specificity). It is the factor by which a positive result multiplies the ODDS of the condition, and it contains no prevalence term at all — so it can be carried from the study population to yours.

LR− = (1 − sensitivity) ÷ specificity does the same for a negative result. Both are verified in the suite to be unchanged when the same test is applied at five different prevalences.

The rough scale is worth memorising. An LR+ above 10 moves a diagnosis decisively; 5 to 10 moderately; 2 to 5 slightly; below 2 barely at all. For LR−, below 0.1 rules out decisively and above 0.5 barely helps.

Using them means working in odds, not probabilities. Convert your pre-test probability to odds, multiply by the likelihood ratio, convert back. The tool does this for you in the prevalence panel, which is the same calculation with the conversion hidden.

And a test with an LR+ near 1 adds nothing whatever its sensitivity looks like. A test that is 99% sensitive and 50% specific has an LR+ of 1.98 — it barely moves a diagnosis, despite the impressive-sounding first number.

Accuracy is the measure a useless test can win

A single headline number for a classifier is always tempting and accuracy is the worst candidate for the job.

At 1% prevalence, a test that always answers “negative” is 99% accurate. It detects nothing. Its sensitivity is zero, its PPV is undefined, and its accuracy beats most real tests.

Balanced accuracy refuses that. It averages sensitivity and specificity, so the always-negative test scores exactly 50% — which is what a coin scores, correctly.

Youden’s J is the same idea shifted: sensitivity + specificity − 1, so 0 is worthless and 1 is perfect. It is also the quantity maximised when choosing a threshold from an ROC curve, which makes it the natural summary when a cut-off is being selected.

The Matthews correlation coefficient is the strictest. It uses all four cells and runs from −1 to 1, and it cannot be fooled by any single cell being large. The always-negative test scores 0 on it.

F1 has its own blind spot, in the opposite direction. It ignores true negatives entirely, which makes it useful when the negative class is uninteresting — information retrieval — and misleading when correctly clearing well people is the point of the test.

The tool flags the combination that gives it away: high accuracy with a low Youden’s J means the number is describing your class imbalance rather than your test.

Rule in or rule out: the threshold decides which

Sensitivity and specificity trade against each other along a threshold, and which end you want depends on what a wrong answer costs.

A sensitive test is a rule-out test. It catches nearly everyone who has the condition, at the price of flagging many who do not, so a NEGATIVE result is informative and a positive one needs confirming. Screening tests are built this way deliberately.

A specific test is a rule-in test. It is rarely wrong when positive and misses a lot, so a POSITIVE result confirms and a negative one settles nothing. Confirmatory tests are built this way.

The two presets in the tool show both with the same total sample, and the difference in what each result means is larger than the difference in the headline percentages.

Moving the threshold moves both at once and in opposite directions. That trade-off traced out is the ROC curve, and the area under it summarises the test across every threshold rather than at the one chosen — useful for comparing tests, useless for deciding what to do with a particular result.

The right threshold is a decision about costs, not a statistic. Maximising Youden’s J weights a false positive and a false negative equally, which is almost never true in practice. Where the two errors have different consequences, the optimum moves and no calculator can find it for you.

Reporting a diagnostic evaluation

Five things, and the first two travel while the others do not.

Report sensitivity and specificity with confidence intervals. They are proportions from finite samples, and a sensitivity of 95% from 20 positive cases is much less certain than the same figure from 2,000. Use a Wilson interval, not a Wald one.

Report the likelihood ratios. They are what a reader in a different setting can actually use, and they are omitted far more often than they are included.

Report predictive values WITH the prevalence they came from. A PPV without its prevalence is a number about your sample that reads as a number about the test.

Say how the true status was established. Every figure here depends on a reference standard, and if that standard is itself imperfect the sensitivity and specificity are biased in ways no interval reflects.

And say where the sample came from. A test validated on obvious cases and healthy controls will look far better than it performs on the ambiguous patients it is meant to help — spectrum bias, and it is not visible anywhere in the 2×2 table.

Sources and methodology

References for diagnostic accuracy measures.

• Altman & Bland, “Diagnostic tests 2: predictive values”, BMJ 309, 1994
• Chicco & Jurman on MCC versus accuracy and F1 on imbalanced data, BMC Genomics 21, 2020
• NIST/SEMATECH e-Handbook — classification error rates and their interpretation
• Conditional Probability Calculator — the direction reversal underneath all of this
• Bayes’ Theorem Calculator — the identity that converts sensitivity into PPV
• How we calculate — methodology

Method. Predictive values at an arbitrary prevalence are computed from Bayes’ theorem rather than rescaled from the observed table, so the sample’s own prevalence never leaks into the answer. The suite pins the headline figures exactly — a 99%/99% test gives a PPV of exactly 50% at 1% prevalence and 9.02% at 0.1% — and verifies as properties that PPV rises monotonically with prevalence across 199 points while NPV falls, and that both likelihood ratios are unchanged when the same test is applied at five different prevalences. The accuracy trap is asserted numerically: a test that never returns positive scores 99% accuracy at 1% prevalence with a sensitivity, a Youden’s J and an F1 of exactly zero. That engine is verified on every change against 75 assertions. The count and the per-case breakdown are published on the formula verification page.

Related calculators

Where this goes next:

More in Math, or browse all calculators.