Brier Score
Brier Score (Mean Squared Error) · Also known as: Mean Squared Probability Error
The Brier score measures the mean squared difference between predicted probabilities and actual binary outcomes. It is a simple, interpretable metric for evaluating the accuracy of probabilistic predictions, particularly in weather forecasting and medical diagnosis.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Brier score when you need a simple, interpretable metric for evaluating probabilistic predictions. It is particularly suitable for weather forecasting, medical risk assessment, and other domains where probability calibration matters. Use it as an alternative to log-loss when you prefer a metric proportional to mean squared error.
Strengths & limitations
- Simple and intuitive: directly measures squared error of probability predictions.
- Interpretable: Brier score of 0 = perfect, 0.25 = random on balanced data.
- Suitable for evaluating calibration of probabilistic forecasts.
- Computable for both binary and multi-class classification.
- Less sensitive than log-loss to very confident wrong predictions.
- Assumes squared error is the appropriate loss function, not always valid.
- Does not directly optimize for classification accuracy.
- Can obscure differences between models if both are poorly calibrated.
Frequently asked
How does Brier score differ from log-loss?
Both evaluate probabilistic predictions, but they emphasize different aspects. Brier score is based on squared error, treating all errors equally (an error of 0.2 contributes 0.04). Log-loss uses logarithms, heavily penalizing confident wrong predictions. Log-loss is more commonly used in machine learning, but Brier score is simpler to interpret.
What is a good Brier score?
A Brier score of 0 is perfect. A score of 0.25 represents random guessing on balanced binary problems. On imbalanced datasets, the random baseline differs. In practice, scores below 0.1 are usually considered very good, 0.1-0.2 is good, 0.2-0.3 is moderate, and above 0.3 is poor.
Can Brier score be used for multi-class problems?
Yes. For K classes, compute the Brier score as 1/N * sum over all samples and classes of (p_ij - y_ij)^2, where p_ij is the predicted probability for class j of sample i, and y_ij is 1 if sample i is class j, 0 otherwise. This naturally extends the binary formulation.
Sources
- Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1-3. DOI: 10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2 ↗
- Murphy, A. H. (1973). A new vector partition of the probability score. Journal of Applied Meteorology, 12(4), 595-600. DOI: 10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2 ↗
How to cite this page
ScholarGate. (2026, June 3). Brier Score (Mean Squared Error). ScholarGate. https://scholargate.app/en/model-evaluation/brier-score
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- AccuracyModel Evaluation↔ compare
- Log-Loss (Cross-Entropy Loss)Model Evaluation↔ compare
- Mean Absolute ErrorModel Evaluation↔ compare