Why a probability is not forecast skill
How raw ensemble probabilities, reliability and skill differ. Why a 60% forecast needs verification against observations before it can be treated as calibrated.
Probability, reliability and skill
A probability answers “what share of this model’s members fall in this category?”. Skill asks whether forecasts perform better than a reference such as climatology, under a chosen score. Reliability asks whether events forecast at 60% occur about 60% of the time across many comparable cases. Both require verification against observations; a single probability map establishes neither.
Why the distinction matters
Seasonal forecast skill varies enormously by region, season, variable and lead. The same upper-tercile probability in two regions need not have the same reliability or skill. Those differences must be assessed for the relevant system, event, season and lead, rather than inferred from the forecast colour.
Reliability
For a reliable forecast, an event assigned 60% occurs in about 60% of comparable cases. That is an event frequency, not a general “percent correct” score. Raw ensemble probabilities can be over- or underconfident. Ensemble spread need not match forecast error, and shared model errors are not fully sampled by the members. Centres publish reliability diagrams and skill scores for their systems; the C3S site provides verification maps for each contributing model. Consult those before treating any probability here as a calibrated confidence.
What this site does and does not provide
This site derives probabilities directly from member counts, with empirical thresholds from the 1993–2016 hindcast, and applies no additional observation-based probability calibration. It does not yet compare forecasts with observations, and it presents no skill scores. Agreement between several models is a useful qualitative signal but is not a substitute for verification, and “neutral” on a tercile summary is not the same as a confident forecast of near-normal conditions.
Read probability maps as a description of what a model’s ensemble contains. Combine them with published skill assessments, with agreement across the five models here, and with consistency across runs before acting on them.
Sources and method
These references explain the underlying science and provider conventions. Descriptions of this site’s calculations, colours and coverage were checked against its processing code and imported data.