Loading…
DATA STATUS
457 series loaded · last import on 2026-10-26

Forecast verification Each forecast sealed before the event, then scored

The one test that cannot be arranged after the fact. Every forecast of the Forecasts page is written into a sealed log when it is issued, together with the reference forecasts of the same moment (climatology, persistence and, for storms, the official NOAA forecast issued before the window opens). Once the window has closed, it is compared with what actually happened. The scores below only cover these forecasts, never data the models could have seen. The same principle applied to the links themselves: registered hypotheses.

Geomagnetic storm

today’s forecast

2 forecasts logged: 0 checked, 2 waiting for their window to close, 0 without complete observations.

Detection horizon: about 1,959 checked forecasts are needed to show skill over climatology (5 % risk, 80 % power), if the forecasts issued are reliable. 0 are checked so far.

No forecast checked yet: the first windows have not closed.

Radiation-belt electrons

today’s forecast

2 forecasts logged: 0 checked, 2 waiting for their window to close, 0 without complete observations.

Detection horizon: about 3,011 checked forecasts are needed to show skill over climatology (5 % risk, 80 % power), if the forecasts issued are reliable. 0 are checked so far.

No forecast checked yet: the first windows have not closed.

Aftershocks of recent large earthquakes

today’s forecast

47 forecasts logged: 0 checked, 47 waiting for their window to close, 0 without complete observations.

No forecast checked yet: the first windows have not closed.

Global temperature of the coming months

today’s forecast

4 forecasts logged: 0 checked, 4 waiting for their window to close, 0 without complete observations.

No forecast checked yet: the first windows have not closed.

Integrity of the log

Forecasts and hypotheses are written into two chains of sealed entries: each seal is the SHA-256 digest of the previous seal and of the entry’s canonical text. Every day, a short text naming the head of each chain is stamped in the Bitcoin blockchain by the public OpenTimestamps calendars. Anyone who downloaded an anchor can check that the log of that day existed in that form, independently of this site.

ChainEntriesHead (last seal)StateDownload
Forecast log 56 (of which 55 sealed afterwards) a1ad70c78aafbdb1… chain intact JSON · CSV
Hypothesis register 5 4b1f1c7950e8462a… chain intact JSON

Daily anchors (1)

Check it yourself

The chain is checked in a few lines (Python below): recompute each seal from the previous one and the entry’s text (field record), and compare the head with an anchor’s. The anchor is checked with the reference OpenTimestamps client: download anchor-YYYYMMDD.txt and its .ots into the same folder, then ots verify. The Bitcoin stamp is confirmed a few hours after the anchor: ots upgrade then completes the proof.

import hashlib, json, urllib.request
chain = json.load(urllib.request.urlopen("https://observatory.lelama.digital/api/ledger/forecasts.json"))
seal = ""
for entry in chain["entries"]:
    seal = hashlib.sha256((seal + "\n" + entry["record"]).encode()).hexdigest()
    assert seal == entry["seal"], entry["id"]
print("chain intact, head", seal == chain["head"], seal)

# pip install opentimestamps-client, then, once Bitcoin has confirmed the stamp (a few hours):
#   ots upgrade anchor-YYYYMMDD.txt.ots && ots verify anchor-YYYYMMDD.txt.ots
Method and sources

Log. For each event and window, only the first forecast issued is kept, with its time of issue and the references of the same instant. Each entry is sealed: its canonical form (JSON, sorted keys) is chained to the previous one by SHA-256, and the head of the chain is stamped daily in the Bitcoin blockchain through OpenTimestamps. An entry changed, removed or back-dated breaks the chain.

Scores. Brier score (Brier, 1950) and its decomposition (Murphy, 1973) over the distinct probability values; skill over each reference on the forecasts they share; 95 % moving-block bootstrap interval (Künsch, 1989), blocks as long as the overlap of the windows (three days for storms issued daily); ROC area; reliability diagram with consistency ranges (Bröcker & Smith, 2007). For temperature: bias, root-mean-square error, CRPS of a normal distribution (Gneiting et al., 2005), skill over persistence, coverage of the 90 % interval.

Detection horizon. If every forecast p were reliable, the score difference d = (b − o)² − (p − o)² with climatology b would have mean (p − b)² and variance 4(p − b)² p(1 − p): this gives the number of independent forecasts needed (one-sided test at 5 %, 80 % power), multiplied by the overlap of the windows.

Official forecast. For storms, the NOAA bulletin issued the evening before the window opens: the sum of the probabilities of a minor, moderate and strong storm on a day gives P(Kp ≥ 5) that day, and the three days are combined as independent. This bulletin is issued about 22 hours after the Observatory’s forecast and so has more information.

Reading these data
In the first weeks, the scores rest on a few dozen forecasts and their intervals are wide: a negative or very high skill means nothing then. Three-day windows issued daily overlap, and three successive storm forecasts are worth one independent observation. Aftershock forecasts have no climatological reference: only their score, reliability and ROC area are given. Entries logged before the seal was introduced (25–26 October 2026) were sealed afterwards and are flagged as such.
SOURCES OF THIS PAGE
Brier (1950)
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. · doi:10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
Licence : Published literature; cited
Murphy (1973)
Murphy, A. H. (1973). A new vector partition of the probability score. Journal of Applied Meteorology, 12(4), 595–600. · doi:10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2
Licence : Published literature; cited
Bröcker et al. (2007)
Bröcker, J., & Smith, L. A. (2007). Increasing the reliability of reliability diagrams. Weather and Forecasting, 22(3), 651–661. · doi:10.1175/WAF993.1
Licence : Published literature; cited
Gneiting et al. (2005)
Gneiting, T., Raftery, A. E., Westveld, A. H., & Goldman, T. (2005). Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Monthly Weather Review, 133(5), 1098–1118. · doi:10.1175/MWR2904.1
Licence : Published literature; cited
Künsch (1989)
Künsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. The Annals of Statistics, 17(3), 1217–1241. · doi:10.1214/aos/1176347265
Licence : Published literature; cited
Jolliffe et al. (2011)
Jolliffe, I. T., & Stephenson, D. B. (Eds.) (2011). Forecast Verification: A Practitioner’s Guide in Atmospheric Science, 2nd ed. Wiley. · doi:10.1002/9781119960003
Licence : Published literature; cited
Wilks (2019)
Wilks, D. S. (2019). Statistical Methods in the Atmospheric Sciences, 4th ed., ch. 9. Elsevier. · doi:10.1016/C2017-0-03921-6
Licence : Published literature; cited