R/custom_model_validation.R
custom_model_examples_template.RdReturn a small example table in the format create_custom_model() expects
for validation_examples. Each example value groups one worked case:
history rows carry observed target values, future rows leave the target
missing and carry the independently calculated expected forecast. Replace
the illustrative values with cases calculated from your own business rule.
custom_model_examples_template(forecast_horizon = 3, date_type = "month")Number of future rows per example. Must match the
forecast horizon used by create_custom_model().
Date cadence: "year", "quarter", "month", "week"
or "day". Must match the cadence used by create_custom_model().
A data frame with columns example, Date, Target, expected
and how. The template holds one normal example (last_value) and one
edge case (edge_zero_history) for an illustrative "repeat the last
observed value" rule.
The table may also contain:
Series columns named like the forecast's combo_variables. Omit
them for one-series examples. An example with two or more series
validates global logic; one series validates local logic.
Driver columns named like external_regressors, filled on every
row, including future rows.
edge_case (logical). Example names starting with edge_ are
edge cases automatically.
expected_error, written as "fit:code" or "predict:code", for
an edge case whose model must stop with that error code. Leave
expected missing on its future rows.
how, a short description of how the expected values were
calculated, and tolerance, an absolute tolerance up to 0.01.
The target column may be named Target or like the forecast's
target_variable. Supply two to eight examples including at least one
edge case. Examples are visible to the LLM; use nonconfidential values.
custom_model_examples_template()
#> example Date Target expected how
#> 1 last_value 2024-01-01 100 NA Repeat the last observed value.
#> 2 last_value 2024-02-01 110 NA Repeat the last observed value.
#> 3 last_value 2024-03-01 120 NA Repeat the last observed value.
#> 4 last_value 2024-04-01 NA 120 Repeat the last observed value.
#> 5 last_value 2024-05-01 NA 120 Repeat the last observed value.
#> 6 last_value 2024-06-01 NA 120 Repeat the last observed value.
#> 7 edge_zero_history 2024-01-01 0 NA All-zero history repeats zero.
#> 8 edge_zero_history 2024-02-01 0 NA All-zero history repeats zero.
#> 9 edge_zero_history 2024-03-01 0 NA All-zero history repeats zero.
#> 10 edge_zero_history 2024-04-01 NA 0 All-zero history repeats zero.
#> 11 edge_zero_history 2024-05-01 NA 0 All-zero history repeats zero.
#> 12 edge_zero_history 2024-06-01 NA 0 All-zero history repeats zero.
custom_model_examples_template(forecast_horizon = 2, date_type = "quarter")
#> example Date Target expected how
#> 1 last_value 2024-01-01 100 NA Repeat the last observed value.
#> 2 last_value 2024-04-01 110 NA Repeat the last observed value.
#> 3 last_value 2024-07-01 120 NA Repeat the last observed value.
#> 4 last_value 2024-10-01 NA 120 Repeat the last observed value.
#> 5 last_value 2025-01-01 NA 120 Repeat the last observed value.
#> 6 edge_zero_history 2024-01-01 0 NA All-zero history repeats zero.
#> 7 edge_zero_history 2024-04-01 0 NA All-zero history repeats zero.
#> 8 edge_zero_history 2024-07-01 0 NA All-zero history repeats zero.
#> 9 edge_zero_history 2024-10-01 NA 0 All-zero history repeats zero.
#> 10 edge_zero_history 2025-01-01 NA 0 All-zero history repeats zero.