Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Benchmarks

Query station benchmark metrics for model evaluation and comparison.

Query station benchmark metrics

post

Query benchmark metrics (RMSE, MAE, bias, CRPS, quantiles) for forecast models evaluated against weather station observations.

CRPS (Continuous Ranked Probability Score) is derived from the per-ensemble-member forecast errors; for deterministic models it equals MAE.

Metric selection: By default all metrics are computed and returned. Pass metrics (any of rmse, mae, bias, crps) to compute and return only those. Requesting only mean-based metrics (rmse/mae/bias) is significantly faster for ensemble models, since CRPS is the only metric that needs the per-ensemble-member distribution.

Useful for:

  • Comparing forecast model performance

  • Evaluating model accuracy by region

  • Analyzing forecast errors over time

Multi-Model Support:

  • Query multiple models in a single request to compare performance

  • Results include model name in each row for easy comparison

Geographic Filtering:

  • market_zone: Filter by energy market zone (e.g., "DE", "FR")

  • country_key: Filter by country code (e.g., "DE", "US")

Note: Either station_ids or geo must be provided (mutually exclusive).

Authentication: Requires API key.

For more information on benchmark metrics, see docs.jua.ai.

Authorizations
Query parameters
formatstring · enumOptional

Response format: 'json' for columnar JSON or 'arrow' for Apache Arrow IPC stream

Default: jsonPossible values:
include_unitsbooleanOptional

When true, JSON responses are wrapped in {data, units}.

Default: false
Body

Query parameters for station benchmark data.

station_idsstring[] · nullableOptional

List of specific station IDs to query. Mutually exclusive with geo filter.

start_timestring · date-timeRequired

Start time for benchmark period

end_timestring · date-timeRequired

End time for benchmark period

max_prediction_timedelta_minutesintegerOptional

Maximum prediction lead time in minutes

Default: 28800
debiasbooleanOptional

If True, subtract a per-(model, valid-time ISO week, valid-time hour) bias estimate from each error before computing RMSE / MAE / bias. The bias is computed on the fly from synoptic_station_bias_europe (an AggregatingMergeTree of the trailing four weeks' errors over European stations, refreshed once per week). Affects the air_temperature_2m, wind_speed_10m and surface_downwelling_shortwave_flux_sum_1h (solar radiation) metrics; for solar the per-hour bias is subtracted before the hourly errors are accumulated into step windows.

Default: false
init_hoursinteger[] · nullableOptional

Filter to forecasts initialised at the given UTC hours. Each value must be in 0..23 (e.g. [0, 12] keeps only 00Z and 12Z runs). None or empty list = all hours (no filtering).

restrict_init_datesstring · date[] · nullableOptional

Internal: restrict the query to these forecast init dates (toDate(init_time)). Set by the per-model fan-out path so each single-model sub-query is scored over the SAME set of days that the multi-model intersection would have selected (the intersection is computed once up front and injected here). Not part of the public API surface; leave unset for normal single-query execution.

solar_step_max_overrideinteger · nullableOptional

Internal: pin the solar accumulation window (S_max, minutes) to the value computed from the ORIGINAL multi-model request. The per-model fan-out sets this so every single-model sub-query accumulates to the SAME window (the coarsest native step across all requested models) -- otherwise a fine model would report hourly RMSE while a coarse one reports 6h-window RMSE in the same chart, which isn't comparable. Not part of the public API surface; leave unset for normal execution.

obs_bucketsbooleanOptional

If True, additionally stratify RMSE / MAE / bias by the observed value's distribution. Each metric is reported per observed-value bucket: < P5, P5-P25, P25-P75, P75-P95, > P95 (plus an 'all' bucket). The P5/P25/P75/P95 thresholds are computed once over the selected stations and time window so the buckets are identical across models and lead times. This recovers the observed value by joining synoptic_station_error back to the station observation table, so it is heavier than the default aggregation and REQUIRES exactly one variable.

Default: false
Responses
200

Successfully retrieved benchmark metrics

anyOptional
post/v1/station-benchmarks/metrics

Get available benchmark dates

get

Retrieve available dates for station benchmark data with optional filtering.

Useful for:

  • Discovering available benchmark data

  • Finding recent benchmark dates

  • Planning benchmark queries

Authentication: Requires API key.

For more information, see docs.jua.ai.

Authorizations
Query parameters
modelstring · nullableOptional

Optional model name to filter by

days_lookbackinteger · min: 1 · max: 365Optional

Number of days to look back from today

Default: 365
Responses
200

Successfully retrieved available dates

application/json

Response containing available benchmark dates.

datesstring[]Required
get/v1/station-benchmarks/available-dates

Get solar-benchmark geographic coverage

get

List the country and market-zone codes that have at least one audited solar radiation station.

The surface-solar-radiation benchmark is scored only over stations that passed the offline radiation quality audit (a much smaller set than the synoptic network). This endpoint returns the regions where a solar country / market-zone benchmark can actually be computed, so the dashboard can restrict the solar-benchmark region pickers to non-empty selections.

Authentication: Requires API key.

Authorizations
Responses
200

Successfully retrieved solar coverage

application/json

Country / market-zone codes covered by the clean solar-station set.

country_keysstring[]Required
market_zonesstring[]Required
get/v1/station-benchmarks/solar-coverage
PreviousForecastNextEntsoe

Last updated