Space-Time Forecast Calibration

Comparing a transformer with simpler forecast corrections using synthetic data.

Why it matters

Comparing a complex forecast model with simpler corrections helps make its behavior easier to assess. This benchmark demonstrates that comparison using synthetic data.

My contribution

I built a synthetic-data benchmark that compares statistical corrections, a NumPy reference, and a PyTorch transformer.

Result

The repository and demo page show the experiment setup and results.

Tools

Python
NumPy
PyTorch
Transformers
Forecast calibration

The results use synthetic data, not real-world climate tests; data-loading code does not show operational forecast accuracy.

Project gallery

Synthetic demonstration figures; these do not show field performance. 9 images. Select an image to view it full size.

Synthetic benchmark: forecast and reference maps

Synthetic benchmark: RMSE by forecast lead

Synthetic benchmark: probabilistic skill by lead

Synthetic benchmark: reliability diagram

Synthetic benchmark: ensemble rank histogram

Synthetic benchmark: ensemble spread and error

Synthetic benchmark: extreme-event Brier skill

Synthetic benchmark: tail-distribution comparison

Technical details

Independent portfolio demonstration

A complex forecast model needs to be compared with simpler methods.

Create a synthetic forecast-correction task.
Compare the transformer with statistical corrections.
Publish the tests, measurements, and results.

← Back to projects