Checks a prepared input table against the rules prep_data() enforces and
reports how preparation will treat the data, without starting a run or
writing anything to disk.
validate_prepared_input(
input_data,
combo_variables,
target_variable,
date_type,
forecast_horizon,
external_regressors = NULL,
hist_start_date = NULL,
hist_end_date = NULL,
combo_cleanup_date = NULL,
fiscal_year_start = 1,
clean_missing_values = TRUE,
forecast_approach = "bottoms_up",
parallel_processing = NULL,
back_test_scenarios = NULL,
back_test_spacing = NULL
)A standard data frame, tibble, or spark data frame using sparklyr of historical time series data.
Can also include external regressors for both historical and future data.
Leading and trailing whitespace in character combo-variable values is
removed before Finn creates internal series identifiers. Values left blank
are treated as missing; missing combo values form the "NA" label in
series names. A combo column with both missing values and the text "NA"
gives both the same label, so bottoms-up forecasts combine them and
standard and grouped hierarchies stop with an error.
List of column headers within input data to be used to separate individual time series.
The column header formatted as a character value within input data you want to forecast.
The date granularity of the input data. Finn accepts the following as a character string: day, week, month, quarter, year.
Number of periods to forecast into the future.
List of column headers within input data to be used as features in multivariate models. Controls built-in R1/R2 recipes only. C1 retains eligible original numeric, character, factor, logical and Date columns independently of this list in bottoms_up runs; hierarchical C1 nodes keep only these aggregated regressors. Each custom workflow uses only its declared predictors plus Date and Combo.
Date value of when your input_data starts. Default of NULL uses earliest date value in input_data.
Date value of when your input_data ends. Default of NULL uses the latest date value in input_data.
Date value to remove individual time series that don't contain non-zero values after that specified date. Default of NULL is to not remove any time series and attempt to forecast all time series.
Month number of start of fiscal year of input data, aids in building out date features. Formatted as a numeric value. Default of 1 assumes fiscal year starts in January.
If TRUE, cleans missing values. Only impute values for missing data within an existing series, and does not add new values onto the beginning or end, but does provide a value of 0 for said values.
Forecast approach, as passed to prep_data().
It only changes R12, which builds the hierarchy's series names and maps
external regressors for standard and grouped hierarchies. Every approach
expects bottom-level series, so totals mixed with their parts fail P7
for all.
Default of NULL runs no parallel processing and forecasts each individual time series one after another. Value of 'local_machine' leverages all cores on current machine Finn is running on. Value of 'spark' runs time series in parallel on a spark cluster in Azure Databricks/Synapse.
Number of back-test scenarios, as passed to
prep_models(). Default of NULL chooses them from the history length.
Periods between back-test scenarios, as passed to
prep_models(). Default of NULL uses the date type default.
A list with:
passed: TRUE when no check failed.
checks: tibble with one row per rule and columns Rule,
Description, Status ("pass", "fail", "report" or
"skipped") and Message.
summary: list with the history start and end dates, row, series,
history and future row counts, date_type and forecast_horizon.
details: list of tibbles and vectors behind the preparation checks:
future_regressors, future_regressor_coverage, gaps,
target_values, cleanup_combos, total_like_values,
aggregate_levels, splits and short_history_combos.
Input rules R1-R8 and R10-R12 reuse the checks in prep_data(),
so a failure here carries the same message prep_data() would raise. R9
is stricter than prep_data(): it also checks that date_type matches the
spacing of the dates. Preparation checks
(P1-P8) describe what prep_data() and the train/test split planning
will do with the data, such as future rows, padded gaps, missing values,
series removed by combo_cleanup_date and short histories. A check fails
only when the data cannot be prepared correctly; behavior that is applied
automatically is returned with status "report".
Rules:
R1 combo variables, the target and external regressors exist.
R2 Date is not a combo variable, the target or an external regressor.
R3 the date column is named Date and has class Date.
R4 combo variables are not date columns.
R5 the target is numeric.
R6 reports combo columns with values that are blank after trimming
whitespace. They are treated as missing values and form the "NA" label
in series names. Always passes.
R7 each combo and Date pair appears once.
R8 month, quarter and year dates fall on one day of the month.
R9 date_type is valid and is not finer or coarser than the data.
R10 fiscal_year_start is a number from 1 to 12.
R11 Spark tables run with Spark parallel processing.
R12 standard and grouped hierarchies can name every series and map every
external regressor: no combo column mixes missing or blank values with the
text "NA", no grouped numeric combo column mixes NA with NaN, no two
series map to the same name after name cleanup, at least two series remain
after combo_cleanup_date, and no external regressor has NaN, infinite
or conflicting values or an ambiguous source level. Always passes for
bottoms_up.
P1 history end date and the rows treated as future periods.
P2 external regressors with future values and their coverage.
P3 missing periods that preparation pads.
P4 missing, NaN and infinite target values.
P5 weekly dates share a weekday; quarterly and yearly dates share a
period month.
P6 series removed by combo_cleanup_date.
P7 total or subtotal series mixed with the series they sum.
P8 history supports the forecast horizon and back-test splits.
Preparation checks run only when every input rule passes, and a check that
depends on a failed check is skipped. Spark tables are collected locally
after R11 is checked.
data_tbl <- timetk::m4_monthly %>%
dplyr::rename(Date = date) %>%
dplyr::mutate(id = as.character(id)) %>%
dplyr::filter(Date >= "2010-01-01")
result <- validate_prepared_input(
input_data = data_tbl,
combo_variables = "id",
target_variable = "value",
date_type = "month",
forecast_horizon = 3
)
result$passed
#> [1] TRUE
result$checks
#> # A tibble: 20 × 4
#> Rule Description Status Message
#> <chr> <chr> <chr> <chr>
#> 1 R1 Required columns exist pass Passed.
#> 2 R2 Date is not used as a combo variable, target or regress… pass Passed.
#> 3 R3 Date column is named Date and has class Date pass Passed.
#> 4 R4 Combo variables are not dates pass Passed.
#> 5 R5 Target is numeric pass Passed.
#> 6 R6 Blank combo values are treated as missing pass Passed.
#> 7 R7 Each combo and Date pair appears once pass Passed.
#> 8 R8 Period dates fall on the same day of the month pass Passed.
#> 9 R9 date_type is valid and matches the data frequency pass 260 of…
#> 10 R10 fiscal_year_start is a number from 1 to 12 pass Passed.
#> 11 R11 Spark tables use Spark parallel processing pass Passed.
#> 12 R12 Hierarchy series and regressors can be prepared pass Not ap…
#> 13 P1 History end date and future rows pass Histor…
#> 14 P2 Future external regressor values pass No ext…
#> 15 P3 Missing periods padded during preparation pass No mis…
#> 16 P4 Missing and non-finite target values pass No mis…
#> 17 P5 Dates align to the date_type grid pass Passed.
#> 18 P6 Series removed by combo_cleanup_date pass combo_…
#> 19 P7 No total series mixed with the series they sum pass No tot…
#> 20 P8 History supports the forecast horizon and back tests pass 13 bac…