Checks a prepared input table against the rules prep_data() enforces and
reports how preparation will treat the data, without starting a run or
writing anything to disk.
validate_prepared_input(
input_data,
combo_variables,
target_variable,
date_type,
forecast_horizon,
external_regressors = NULL,
hist_start_date = NULL,
hist_end_date = NULL,
combo_cleanup_date = NULL,
fiscal_year_start = 1,
clean_missing_values = TRUE,
forecast_approach = "bottoms_up",
parallel_processing = NULL,
back_test_scenarios = NULL,
back_test_spacing = NULL
)A standard data frame, tibble, or spark data frame using sparklyr of historical time series data. Can also include external regressors for both historical and future data. Leading and trailing whitespace in character combo-variable values is removed before Finn creates internal series identifiers.
List of column headers within input data to be used to separate individual time series.
The column header formatted as a character value within input data you want to forecast.
The date granularity of the input data. Finn accepts the following as a character string: day, week, month, quarter, year.
Number of periods to forecast into the future.
List of column headers within input data to be used as features in multivariate models. Controls built-in R1/R2 recipes only. C1 retains eligible original numeric, character, factor, logical and Date columns independently of this list in bottoms_up runs; hierarchical C1 nodes keep only these aggregated regressors. Each custom workflow uses only its declared predictors plus Date and Combo.
Date value of when your input_data starts. Default of NULL uses earliest date value in input_data.
Date value of when your input_data ends. Default of NULL uses the latest date value in input_data.
Date value to remove individual time series that don't contain non-zero values after that specified date. Default of NULL is to not remove any time series and attempt to forecast all time series.
Month number of start of fiscal year of input data, aids in building out date features. Formatted as a numeric value. Default of 1 assumes fiscal year starts in January.
If TRUE, cleans missing values. Only impute values for missing data within an existing series, and does not add new values onto the beginning or end, but does provide a value of 0 for said values.
Forecast approach, as passed to prep_data().
It is validated but does not change the checks: every approach expects
bottom-level series, so totals mixed with their parts fail P7 for all.
Default of NULL runs no parallel processing and forecasts each individual time series one after another. Value of 'local_machine' leverages all cores on current machine Finn is running on. Value of 'spark' runs time series in parallel on a spark cluster in Azure Databricks/Synapse.
Number of back-test scenarios, as passed to
prep_models(). Default of NULL chooses them from the history length.
Periods between back-test scenarios, as passed to
prep_models(). Default of NULL uses the date type default.
A list with:
passed: TRUE when no check failed.
checks: tibble with one row per rule and columns Rule,
Description, Status ("pass", "fail", "report" or
"skipped") and Message.
summary: list with the history start and end dates, row, series,
history and future row counts, date_type and forecast_horizon.
details: list of tibbles and vectors behind the preparation checks:
future_regressors, future_regressor_coverage, gaps,
target_values, cleanup_combos, total_like_values,
aggregate_levels, splits and short_history_combos.
Input rules R1-R8, R10 and R11 reuse the checks in prep_data(),
so a failure here carries the same message prep_data() would raise. R9
is stricter than prep_data(): it also checks that date_type matches the
spacing of the dates. Preparation checks
(P1-P8) describe what prep_data() and the train/test split planning
will do with the data, such as future rows, padded gaps, missing values,
series removed by combo_cleanup_date and short histories. A check fails
only when the data cannot be prepared correctly; behavior that is applied
automatically is returned with status "report".
Rules:
R1 combo variables, the target and external regressors exist.
R2 Date is not a combo variable, the target or an external regressor.
R3 the date column is named Date and has class Date.
R4 combo variables are not date columns.
R5 the target is numeric.
R6 combo values are not blank after trimming whitespace.
R7 each combo and Date pair appears once.
R8 month, quarter and year dates fall on one day of the month.
R9 date_type is valid and is not finer or coarser than the data.
R10 fiscal_year_start is a number from 1 to 12.
R11 Spark tables run with Spark parallel processing.
P1 history end date and the rows treated as future periods.
P2 external regressors with future values and their coverage.
P3 missing periods that preparation pads.
P4 missing, NaN and infinite target values.
P5 weekly dates share a weekday; quarterly and yearly dates share a
period month.
P6 series removed by combo_cleanup_date.
P7 total or subtotal series mixed with the series they sum.
P8 history supports the forecast horizon and back-test splits.
Preparation checks run only when every input rule passes, and a check that
depends on a failed check is skipped. Spark tables are collected locally
after R11 is checked.
data_tbl <- timetk::m4_monthly %>%
dplyr::rename(Date = date) %>%
dplyr::mutate(id = as.character(id)) %>%
dplyr::filter(Date >= "2010-01-01")
result <- validate_prepared_input(
input_data = data_tbl,
combo_variables = "id",
target_variable = "value",
date_type = "month",
forecast_horizon = 3
)
result$passed
#> [1] TRUE
result$checks
#> # A tibble: 19 × 4
#> Rule Description Status Message
#> <chr> <chr> <chr> <chr>
#> 1 R1 Required columns exist pass Passed.
#> 2 R2 Date is not used as a combo variable, target or regress… pass Passed.
#> 3 R3 Date column is named Date and has class Date pass Passed.
#> 4 R4 Combo variables are not dates pass Passed.
#> 5 R5 Target is numeric pass Passed.
#> 6 R6 Combo values are not blank after trimming pass Passed.
#> 7 R7 Each combo and Date pair appears once pass Passed.
#> 8 R8 Period dates fall on the same day of the month pass Passed.
#> 9 R9 date_type is valid and matches the data frequency pass 260 of…
#> 10 R10 fiscal_year_start is a number from 1 to 12 pass Passed.
#> 11 R11 Spark tables use Spark parallel processing pass Passed.
#> 12 P1 History end date and future rows pass Histor…
#> 13 P2 Future external regressor values pass No ext…
#> 14 P3 Missing periods padded during preparation pass No mis…
#> 15 P4 Missing and non-finite target values pass No mis…
#> 16 P5 Dates align to the date_type grid pass Passed.
#> 17 P6 Series removed by combo_cleanup_date pass combo_…
#> 18 P7 No total series mixed with the series they sum pass No tot…
#> 19 P8 History supports the forecast horizon and back tests pass 13 bac…