Three hundred forecasts that take only ten distinct values — 0.05, 0.15, and so on — which is exactly the case Murphy’s decomposition was derived for. The score is 0.154500 and reliability minus resolution plus uncertainty is 0.154500, with a leftover of 0.000000000000000111. That is machine precision, and it is the check: the three terms are computed from the binned rates and the score from the raw forecasts, so agreement to the last digit means the binning, the calibration term and the base rate are all right. Reliability is 0.006158, resolution 0.101481 and uncertainty 0.249822, giving a skill score of 0.38156 against always forecasting the base rate.
300 forecasts · base rate 0.4867 · 10 bins
Brier score 0.154500, skill 0.38156
Forecasting the base rate on every case would score 0.249822; this scores 0.154500, a 38.16% improvement. Reliability is 0.006158 and resolution 0.101481 — calibration and discrimination, separately.
Brier score
0.154500
base rate would give 0.24982
Skill
0.38156
against forecasting the base rate
Reliability
0.006158
calibration — lower is better
Resolution
0.101481
discrimination — higher is better
The decomposition, and what it leaves over
Murphy’s three terms, their recombination, and the difference from the score
Term
Value
Meaning
Reliability
0.006158
how far each bin’s outcome rate is from what it promised
Resolution
−0.101481
how far the bin rates are from the base rate
Uncertainty
0.249822
the base rate’s own variance, which no forecast can beat
Recombined
0.154500
reliability − resolution + uncertainty
Actual score
0.154500
the mean squared error itself
Left over
1.110e-16
zero to machine precision — the forecasts sit on bin values
The identity is exact only when every forecast takes a single value per bin. With continuous forecasts the decomposition replaces each one with its bin mean, and the leftover is what that replacement costs — 5.415266e-32 of average within-bin spread here. Presentations that show the three terms summing exactly to the score, on continuous forecasts, are rounding the difference away.
Calibration by bin
Each forecast bin with its mean prediction, observed rate and contribution to reliability
Range
Cases
Predicted
Observed
Gap
Share of reliability
0.00 to 0.10
36
0.0500
0.0556
0.0056
0.1%
0.10 to 0.20
35
0.1500
0.1143
-0.0357
2.4%
0.20 to 0.30
31
0.2500
0.1613
-0.0887
13.2%
0.30 to 0.40
28
0.3500
0.2857
-0.0643
6.3%
0.40 to 0.50
20
0.4500
0.6500
0.2000
43.3%
0.50 to 0.60
29
0.5500
0.5862
0.0362
2.1%
0.60 to 0.70
38
0.6500
0.6316
-0.0184
0.7%
0.70 to 0.80
27
0.7500
0.8889
0.1389
28.2%
0.80 to 0.90
25
0.8500
0.8400
-0.0100
0.1%
0.90 to 1.00
31
0.9500
0.9032
-0.0468
3.7%
This is a reliability diagram as a table. A well-calibrated forecast has the predicted and observed columns tracking each other down the page; the Hosmer-Lemeshow test turns the same comparison into a p-value, which adds a verdict and loses the magnitudes.
Log loss on the same forecasts is 0.478742 against a base-rate baseline of 0.692792. Both are proper scoring rules, so both are minimised by honest forecasting; they differ in how harshly they punish confident errors, and the Brier score is bounded at 1 where log loss is not bounded at all.
The uncertainty term depends only on the base rate, so Brier scores from different problems are not comparable. A score of 0.15 is excellent at a 50% base rate and poor at a 5% one. The skill score divides that out, which is why it is the figure to compare across datasets.
Three terms, and the fourth Skill against the base rate Proper scoring rule Not comparable across base rates
What this tool shows
Reliability minus resolution plus uncertainty equals the Brier score exactly — and only when every forecast sits on a single bin value. On the first preset the leftover is 0.000000000000000111, machine precision. On the second, continuous forecasts binned into deciles, it is −0.000415. Most presentations show the three terms summing neatly to the score whatever the forecasts look like. This page prints the difference, because a decomposition that always balances is one nobody is checking.
The Brier score with Murphy’s reliability, resolution and uncertainty terms
The within-bin term the decomposition drops, printed rather than rounded away
A skill score against forecasting the base rate on every case
A calibration table — predicted against observed in every bin, with each bin’s share of the reliability
Log loss on the same forecasts, for comparison between the two proper scoring rules
Why Brier scores from different problems cannot be compared, and what to compare instead
Three terms, and the fourth Skill against the base rate Proper scoring rule Log loss alongside
A Brier score is not comparable across base rates.
Updated 13 September 2026 · Works in any browser, no installation
The Brier score is the mean squared difference between a probability forecast and what actually happened, and Murphy’s decomposition splits it into calibration, discrimination and the irreducible difficulty of the problem. Those three terms answer different questions: whether the numbers mean what they say, whether they separate the cases at all, and how much of the score was unavoidable given the base rate.
At a glance
Formula shown
BS = (1/n)Σ(fᵢ − oᵢ)², with fᵢ the forecast and oᵢ the outcome. Binning the forecasts and writing f̄ₖ, ōₖ for the mean forecast and observed rate in bin k: reliability = Σ(nₖ/n)(f̄ₖ − ōₖ)², resolution = Σ(nₖ/n)(ōₖ − ō)², uncertainty = ō(1 − ō). The identity BS = reliability − resolution + uncertainty is exact when the forecast is constant within each bin; otherwise the within-bin spread is left over.
Scenario support
Scoring probability forecasts of any binary event, comparing two models on the same data, separating a calibration problem from a discrimination problem, and deciding whether a forecast beats simply predicting the base rate.
Educational estimate
Planning support from the values you enter — not professional advice.
The decomposition is exact, and then it is not
The identity is usually stated without conditions. It has one, and the first two presets are chosen so it holds and then fails.
On bin-valued forecasts the leftover is 0.000000000000000111. Machine precision — the three terms really do sum to the score.
On continuous forecasts binned into deciles it is −0.000415. Small, and not zero.
The cause is the spread of forecasts inside each bin, which averages 0.000803 there. The decomposition replaces each forecast with its bin mean, and the replacement costs something.
Which makes the leftover a self-check. The three terms come from the binned rates and the score from the raw forecasts, computed independently. On bin-valued data they must agree exactly, and the verification suite asserts that on thirty generated sets.
What the three terms actually mean
The names are unhelpful and the quantities are not, so it is worth stating each one in terms of a decision you might make.
Reliability is calibration, and lower is better. It asks whether the cases you gave 30% to actually happened 30% of the time. Zero means your numbers mean what they say.
Resolution is discrimination, and higher is better. It asks whether your forecasts vary in a way that tracks the outcome. Zero means you said the same thing about everything.
Uncertainty is the base rate’s own variance, and nothing you do changes it. It is why a Brier score of 0.15 is excellent at a 50% base rate and poor at a 5% one.
The subtraction is the important part. Resolution enters with a minus sign, so discrimination reduces the score and miscalibration raises it — and a model can trade one for the other, which is the fourth preset.
Perfectly calibrated and completely useless
The third preset is the reference point the whole framework is built around, and it is the argument against judging a model on calibration alone.
Every case gets the same forecast: 0.4125, the sample’s own event rate.
Reliability is 1.77e-30 and resolution is exactly 0. Perfect calibration, zero discrimination.
The score, 0.242344, equals the uncertainty to the last digit, and the skill score is zero.
This model would pass any calibration test comfortably, including Hosmer-Lemeshow, and it is worth nothing. Resolution is the term that notices, and a calibration test has no way to see it.
Discrimination and calibration can cancel
The fourth preset is a model that ranks reasonably well and whose numbers are systematically too high, which is the most common real failure.
Resolution is 0.04168 — it does separate the cases. The high-risk group really does have more events than the low-risk one.
Reliability is 0.03942, an order of magnitude worse than the calibrated presets. The promised rates are about half again the delivered ones.
The two nearly cancel and the skill score is 0.0110. Barely better than forecasting the base rate for everybody.
And the remedy is cheap. Miscalibration is fixable after the fact — a recalibration on held-out data moves reliability towards zero without touching resolution, and the skill score would jump. Poor resolution needs a better model.
Compare skill scores, not Brier scores
The uncertainty term depends only on the base rate, which makes raw Brier scores incomparable across problems in a way that is easy to miss.
At a 50% base rate the uncertainty is 0.25. At 5% it is 0.0475, and at 1% it is 0.0099.
So a rare-event forecaster scores well by doing nothing. Predict 1% for everybody on a 1% problem and the Brier score is 0.0099 — better than a genuinely skilful forecast on a balanced problem.
The skill score divides that out: 1 − score/uncertainty, which is zero for the base-rate forecast whatever the base rate is.
It can go negative, which the fourth preset comes close to — a forecast can be worse than useless, and the raw score will not say so.
Why it is safe to optimise
The Brier score is a proper scoring rule, and the practical meaning of that is narrower and more useful than the definition sounds.
The expected score is minimised by forecasting your true belief. Shading a number towards 0 or 1 to look decisive makes your expected score worse.
So it cannot be gamed by strategic over-confidence, which accuracy and F1 both can.
The verification suite checks this directly by scanning the forecast across a grid at seven true probabilities and requiring the minimum to sit at the truth, for Brier and log loss alike.
Both are proper; they differ in how they punish. Brier is bounded at 1, so the worst possible single mistake is limited. Log loss is unbounded, and one confident error can dominate a whole evaluation.
Reporting a Brier score
Four items, and the first is the one without which the number cannot be read at all.
Give the base rate. A Brier score is not comparable to any other without it, and the uncertainty term makes that concrete.
Give the skill score. That is the figure that travels between problems.
Give reliability and resolution separately. A model that ranks well and is badly calibrated needs a different fix from one that is calibrated and ranks poorly, and a single score hides which you have.
Give the bin count if you report the decomposition. The three terms depend on it, and the leftover grows as the bins get wider.
Method. The score is computed from the raw forecasts and the three decomposition terms from the binned rates, which is what makes their difference a meaningful check rather than a tautology. That difference — the within-bin term — is reported rather than absorbed, along with the average spread of forecasts inside their bins that produces it. The verification suite asserts the identity where it is exact: on thirty generated sets whose forecasts take one value per bin, the leftover must be below 1e-12 and the within-bin spread below 1e-24. It separately asserts the anchor case — a flat base-rate forecast must have reliability and resolution of zero and a score equal to the uncertainty — and that the score, the recombination and the leftover satisfy their relation on 120 continuous sets. Properness is checked directly by scanning the forecast across a grid at seven true probabilities and requiring the expected score to be minimised at the truth. That engine is verified on every change against 100 assertions. The count and the per-case breakdown are published on the formula verification page.
Related calculators
Where this goes next:
Log LossLog loss with the Brier score on the same forecasts, the penalty each applies at increasing confidence, and how much of the total a single case carries.
Hosmer-LemeshowThe Hosmer-Lemeshow calibration test with the group count swept from five to fifteen, per-group promised-against-observed rates, and the Brier reliability alongside.
Logistic RegressionLogistic regression with odds ratios converted to risk ratios at your own event rate, a likelihood ratio test, AUC, and separation reported rather than hidden.
ROC Curve & AUCBuilds the curve from raw scores with every threshold enumerated, and computes the AUC twice — trapezoid and Mann-Whitney U — which agree to 1.11e-16 across 300 datasets.
Sensitivity and SpecificitySensitivity, specificity, PPV, NPV, likelihood ratios and MCC from a 2×2 table, with predictive values recomputed across the prevalence range — a 99%/99% test has a PPV of 50% at 1% prevalence and 9% at 0.1%.
Poisson RegressionPoisson regression with rate ratios, the Pearson dispersion check, and quasi-Poisson standard errors printed beside the Poisson ones for every coefficient.
An educational tool. A Brier score depends on the base rate through its uncertainty term, so scores from different problems are not comparable — the skill score is. Murphy’s three-term decomposition is exact only when each forecast takes a single value within its bin; with continuous forecasts a within-bin term is left over, and it is printed here rather than absorbed.