What is the Brier score?
The Brier score is a metric used to evaluate the accuracy of probabilistic predictions, especially for binary outcomes. It measures the mean squared difference between predicted probabilities and the actual outcomes. It is strictly proper, meaning that its expectation is uniquely minimised if the prediction corresponds to the true underlying distribution, and effective, meaning that the expectation is order preserving w.r.t the true probability.
Given a set of probabilistic predictions pi and corresponding observed outcomes yi, the Brier score is defined as:
BS(p, y) = 1/n *Σ (p_i - y_i)^2
References:
Hoessly, L. (2025). On misconceptions about the Brier score in binary prediction models. arXiv. https://arxiv.org/abs/2504.04906