IESO Coincident Peak Prediction — Operational Design & Deployment Architecture

This notebook demonstrates a live prediction workflow using the trained model with current IESO demand data and weather forecasts, and documents the deployment architecture for operational use.

Three-phase daily workflow:

1. Morning forecast (6–7 AM): weather-based demand prediction and risk classification
2. Intraday monitoring (noon–8 PM): real-time demand trajectory tracking
3. Post-day review: accuracy logging and threshold update

Live Prediction Demo

Fetch current weather forecast from Open-Meteo and most recent demand data from the local dataset, then run the trained model to generate today's risk assessment.

Intraday Tracking Visualization

Show how accumulating real-time demand data through the day narrows uncertainty by comparing the current trajectory against historical peak-day demand envelopes.

7-Day Forecast Calendar

API Design Specification

REST API endpoints for an operational deployment of the prediction service.

Endpoint	Method	Description	Response
/predict/today	GET	Morning forecast with risk level	{date, predicted_max_mw, risk_level, window, confidence}
/predict/realtime	GET	Intraday updated probability using actual demand trajectory	{date, hour, current_demand_mw, projected_max_mw, risk_level, trajectory_percentile}
/predict/outlook	GET	7-day forecast outlook	[{date, risk_level, predicted_max_mw, max_temp_c}]
/status/peaks	GET	Current top-5 peaks and displacement threshold	{base_period, peaks: [{rank, date, hour, demand_mw}], threshold_mw}
/status/model	GET	Model metadata and performance	{version, trained_on, test_rmse, last_retrained}
/history/backtest	GET	Walk-forward backtest results	[{base_period, rmse, precision, recall, f1}]
/history/alerts	GET	Alert history for current base period	[{date, alert_level, predicted_mw, actual_mw, was_peak}]

Deployment Architecture

                    IESO Coincident Peak Prediction — System Architecture
                    =====================================================

    ┌─────────────────────┐     ┌──────────────────────┐
    │   IESO Public APIs  │     │  Open-Meteo Weather  │
    │                     │     │                      │
    │  • Hourly Demand    │     │  • 7-Day Forecast    │
    │  • ICI Demand       │     │  • Historical Archive│
    │  • Peak Tracker     │     │                      │
    └────────┬────────────┘     └──────────┬───────────┘
             │                              │
             ▼                              ▼
    ┌─────────────────────────────────────────────────┐
    │              Python Ingestion Layer             │
    │                                                 │
    │  • Fetch & validate data                        │
    │  • Timestamp alignment (HE→datetime)            │
    │  • Gap detection & handling                     │
    │  • Cron: 6 AM daily + hourly noon–8 PM          │
    └────────────────────┬────────────────────────────┘
                         │
                         ▼
    ┌─────────────────────────────────────────────────┐
    │               Feature Engine                    │
    │                                                 │
    │  • Weather → humidex, CDH, rolling averages     │
    │  • Demand → lagged, rolling, momentum           │
    │  • Peak context → threshold tracker             │
    │  • Calendar → holidays, business day flags      │
    └────────────────────┬────────────────────────────┘
                         │
                         ▼
    ┌─────────────────────────────────────────────────┐
    │           XGBoost Prediction Model              │
    │                                                 │
    │  • Daily max demand regression                  │
    │  • RED / YELLOW / GREEN classification           │
    │  • 3-hour peak window estimation                │
    │  • Retrained annually (May 1)                   │
    └────────────────────┬────────────────────────────┘
                         │
                         ▼
    ┌─────────────────────────────────────────────────┐
    │            Flask/FastAPI REST Service            │
    │                                                 │
    │  GET /predict/today    → morning forecast       │
    │  GET /predict/realtime → intraday update        │
    │  GET /predict/outlook  → 7-day outlook          │
    │  GET /status/peaks     → current threshold      │
    └────────────────────┬────────────────────────────┘
                         │
            ┌────────────┼────────────┐
            ▼            ▼            ▼
    ┌──────────┐  ┌──────────┐  ┌──────────┐
    │  Email   │  │   SMS    │  │  Slack   │
    │ (SMTP)   │  │ (Twilio) │  │ Webhook  │
    └──────────┘  └──────────┘  └──────────┘

Model Retraining Schedule

The model should be retrained annually to incorporate the latest base period's data.

Step	Timing	Action
1	May 1	Base period closes. IESO publishes final top-5 peaks.
2	May 1–7	Download completed base period data. Add to training set.
3	May 7–14	Retrain XGBoost on full historical dataset (2010–latest).
4	May 14–21	Validate on previous base period. Compare RMSE and recall to prior model.
5	June 1	Deploy new model for upcoming peak season.
6	Jun–Sep	Monitor prediction accuracy. Flag drift if RMSE > 1.5× historical average.
7	Ongoing	Track structural demand shifts (EV adoption, new industrial loads, BTM solar).

Monitoring & Drift Detection

Key metrics to track during the peak season: