Brier Score
Also known as: Mean Squared Probability Error
The Brier score measures the mean squared difference between predicted probabilities and actual binary outcomes. It is a simple, interpretable metric for evaluating the accuracy of probabilistic predictions, particularly in weather forecasting and medical diagnosis.
Key highlights
- Simple and intuitive: directly measures squared error of probability predictions.
- Interpretable: Brier score of 0 = perfect, 0.25 = random on balanced data.
- Suitable for evaluating calibration of probabilistic forecasts.
- Computable for both binary and multi-class classification.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use Brier score when you need a simple, interpretable metric for evaluating probabilistic predictions. It is particularly suitable for weather forecasting, medical risk assessment, and other domains where probability calibration matters. Use it as an alternative to log-loss when you prefer a metric proportional to mean squared error.
Strengths & limitations
- Simple and intuitive: directly measures squared error of probability predictions.
- Interpretable: Brier score of 0 = perfect, 0.25 = random on balanced data.
- Suitable for evaluating calibration of probabilistic forecasts.
- Computable for both binary and multi-class classification.
- Less sensitive than log-loss to very confident wrong predictions.
- Assumes squared error is the appropriate loss function, not always valid.
- Does not directly optimize for classification accuracy.
- Can obscure differences between models if both are poorly calibrated.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does Brier score differ from log-loss?
Both evaluate probabilistic predictions, but they emphasize different aspects. Brier score is based on squared error, treating all errors equally (an error of 0.2 contributes 0.04). Log-loss uses logarithms, heavily penalizing confident wrong predictions. Log-loss is more commonly used in machine learning, but Brier score is simpler to interpret.
What is a good Brier score?
A Brier score of 0 is perfect. A score of 0.25 represents random guessing on balanced binary problems. On imbalanced datasets, the random baseline differs. In practice, scores below 0.1 are usually considered very good, 0.1-0.2 is good, 0.2-0.3 is moderate, and above 0.3 is poor.
Can Brier score be used for multi-class problems?
Yes. For K classes, compute the Brier score as 1/N * sum over all samples and classes of (p_ij - y_ij)^2, where p_ij is the predicted probability for class j of sample i, and y_ij is 1 if sample i is class j, 0 otherwise. This naturally extends the binary formulation.
Sources
- 1.Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1-3.
- 2.Murphy, A. H. (1973). A new vector partition of the probability score. Journal of Applied Meteorology, 12(4), 595-600.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Brier Score. ScholarGate. https://scholargate.app/model-evaluation/brier-score