Copyright (c) Microsoft Corporation.
Licensed under the MIT License.

This notebook builds a forecasting model using the Prophet algorithm. Prophet is a time series model developed by Facebook that is designed to be simple for non-experts to use, yet flexible and powerful.

Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.

Here, we will use the fable.prophet package which provides a tidyverts frontend to the prophet package itself. As with ETS, prophet does not support time series with missing values, so we again impute them using the ARIMA model forecasts.

srcdir <- here::here("R_utils")
for(src in dir(srcdir, full.names=TRUE)) source(src)

load_objects("grocery_sales", "data_reg.Rdata")
load_objects("grocery_sales", "model_basic.Rdata")

cl <- make_cluster(libs=c("tidyr", "dplyr", "fable", "tsibble", "feasts", "prophet", "fable.prophet"))

oj_modelset_pr <- parallel::clusterMap(cl, function(df, basicmod)
{
    df$logmove <- interpolate(select(basicmod, -c(mean, naive, drift)), df)$logmove
    df %>%
        group_by(store, brand) %>%
        fill(deal:maxpricediff, .direction="downup") %>%
        model(
            pr=prophet(logmove ~ deal + feat + price + maxpricediff),

            pr_tune=prophet(logmove ~ deal + feat + price + maxpricediff +
                growth(n_changepoints=2) + season(period=52, order=5, prior_scale=2)),

            .safely=FALSE
        )
}, oj_trainreg, oj_modelset_basic)

oj_fcast_pr <- parallel::clusterMap(cl, function(mable, newdata, fcast_func)
{
    newdata <- newdata %>%
        fill(deal:maxpricediff, .direction="downup")
    fcast_func(mable, newdata)
}, oj_modelset_pr, oj_testreg, MoreArgs=list(fcast_func=get_forecasts))

destroy_cluster(cl)

save_objects(oj_modelset_pr, oj_fcast_pr,
             example="grocery_sales", file="model_pr.Rdata")

oj_fcast_pr %>%
    bind_rows() %>%
    mutate_at(-(1:3), exp) %>%
    eval_forecasts()

It appears that Prophet does not do better than the simple ARIMA model with regression variables. This is possibly because the dataset does not have a strong time series nature: there is no seasonality, and only weak or nonexistent trends. These are features which the Prophet algorithm is designed to detect, and their absence means that there would be little advantage in using it.

LS0tCnRpdGxlOiBQcm9waGV0IG1vZGVscwpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgpfQ29weXJpZ2h0IChjKSBNaWNyb3NvZnQgQ29ycG9yYXRpb24uXzxici8+Cl9MaWNlbnNlZCB1bmRlciB0aGUgTUlUIExpY2Vuc2UuXwoKYGBge3IsIGVjaG89RkFMU0UsIHJlc3VsdHM9ImhpZGUiLCBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KHRpZHlyKQpsaWJyYXJ5KGRwbHlyKQpsaWJyYXJ5KHRzaWJibGUpCmxpYnJhcnkoZmVhc3RzKQpsaWJyYXJ5KGZhYmxlKQpsaWJyYXJ5KHVyY2EpCmxpYnJhcnkocHJvcGhldCkKIyMgTk9URTogdGhlIGZvbGxvd2luZyBwYWNrYWdlIGNhbiBiZSBpbnN0YWxsZWQgZnJvbSBHaXRIdWIgd2l0aAojIyByZW1vdGVzOjppbnN0YWxsX2dpdGh1YigibWl0Y2hlbGxvaGFyYXdpbGQvZmFibGUucHJvcGhldCIpCmxpYnJhcnkoZmFibGUucHJvcGhldCkKCiMjIyBOT1RFOiBZb3UgbXVzdCBydW4gdGhlIDAyYV9yZWdfbW9kZWxzLlJtZCBub3RlYm9vayBiZWZvcmUgdGhpcyBvbmUKYGBgCgpUaGlzIG5vdGVib29rIGJ1aWxkcyBhIGZvcmVjYXN0aW5nIG1vZGVsIHVzaW5nIHRoZSBbUHJvcGhldF0oaHR0cHM6Ly9mYWNlYm9vay5naXRodWIuaW8vcHJvcGhldC8pIGFsZ29yaXRobS4gUHJvcGhldCBpcyBhIHRpbWUgc2VyaWVzIG1vZGVsIGRldmVsb3BlZCBieSBGYWNlYm9vayB0aGF0IGlzIGRlc2lnbmVkIHRvIGJlIHNpbXBsZSBmb3Igbm9uLWV4cGVydHMgdG8gdXNlLCB5ZXQgZmxleGlibGUgYW5kIHBvd2VyZnVsLgoKPiBQcm9waGV0IGlzIGEgcHJvY2VkdXJlIGZvciBmb3JlY2FzdGluZyB0aW1lIHNlcmllcyBkYXRhIGJhc2VkIG9uIGFuIGFkZGl0aXZlIG1vZGVsIHdoZXJlIG5vbi1saW5lYXIgdHJlbmRzIGFyZSBmaXQgd2l0aCB5ZWFybHksIHdlZWtseSwgYW5kIGRhaWx5IHNlYXNvbmFsaXR5LCBwbHVzIGhvbGlkYXkgZWZmZWN0cy4gSXQgd29ya3MgYmVzdCB3aXRoIHRpbWUgc2VyaWVzIHRoYXQgaGF2ZSBzdHJvbmcgc2Vhc29uYWwgZWZmZWN0cyBhbmQgc2V2ZXJhbCBzZWFzb25zIG9mIGhpc3RvcmljYWwgZGF0YS4gUHJvcGhldCBpcyByb2J1c3QgdG8gbWlzc2luZyBkYXRhIGFuZCBzaGlmdHMgaW4gdGhlIHRyZW5kLCBhbmQgdHlwaWNhbGx5IGhhbmRsZXMgb3V0bGllcnMgd2VsbC4KCkhlcmUsIHdlIHdpbGwgdXNlIHRoZSBmYWJsZS5wcm9waGV0IHBhY2thZ2Ugd2hpY2ggcHJvdmlkZXMgYSB0aWR5dmVydHMgZnJvbnRlbmQgdG8gdGhlIHByb3BoZXQgcGFja2FnZSBpdHNlbGYuIEFzIHdpdGggRVRTLCBwcm9waGV0IGRvZXMgbm90IHN1cHBvcnQgdGltZSBzZXJpZXMgd2l0aCBtaXNzaW5nIHZhbHVlcywgc28gd2UgYWdhaW4gaW1wdXRlIHRoZW0gdXNpbmcgdGhlIEFSSU1BIG1vZGVsIGZvcmVjYXN0cy4KCmBgYHtyfQpzcmNkaXIgPC0gaGVyZTo6aGVyZSgiUl91dGlscyIpCmZvcihzcmMgaW4gZGlyKHNyY2RpciwgZnVsbC5uYW1lcz1UUlVFKSkgc291cmNlKHNyYykKCmxvYWRfb2JqZWN0cygiZ3JvY2VyeV9zYWxlcyIsICJkYXRhX3JlZy5SZGF0YSIpCmxvYWRfb2JqZWN0cygiZ3JvY2VyeV9zYWxlcyIsICJtb2RlbF9iYXNpYy5SZGF0YSIpCgpjbCA8LSBtYWtlX2NsdXN0ZXIobGlicz1jKCJ0aWR5ciIsICJkcGx5ciIsICJmYWJsZSIsICJ0c2liYmxlIiwgImZlYXN0cyIsICJwcm9waGV0IiwgImZhYmxlLnByb3BoZXQiKSkKCm9qX21vZGVsc2V0X3ByIDwtIHBhcmFsbGVsOjpjbHVzdGVyTWFwKGNsLCBmdW5jdGlvbihkZiwgYmFzaWNtb2QpCnsKICAgIGRmJGxvZ21vdmUgPC0gaW50ZXJwb2xhdGUoc2VsZWN0KGJhc2ljbW9kLCAtYyhtZWFuLCBuYWl2ZSwgZHJpZnQpKSwgZGYpJGxvZ21vdmUKICAgIGRmICU+JQogICAgICAgIGdyb3VwX2J5KHN0b3JlLCBicmFuZCkgJT4lCiAgICAgICAgZmlsbChkZWFsOm1heHByaWNlZGlmZiwgLmRpcmVjdGlvbj0iZG93bnVwIikgJT4lCiAgICAgICAgbW9kZWwoCiAgICAgICAgICAgIHByPXByb3BoZXQobG9nbW92ZSB+IGRlYWwgKyBmZWF0ICsgcHJpY2UgKyBtYXhwcmljZWRpZmYpLAoKICAgICAgICAgICAgcHJfdHVuZT1wcm9waGV0KGxvZ21vdmUgfiBkZWFsICsgZmVhdCArIHByaWNlICsgbWF4cHJpY2VkaWZmICsKICAgICAgICAgICAgICAgIGdyb3d0aChuX2NoYW5nZXBvaW50cz0yKSArIHNlYXNvbihwZXJpb2Q9NTIsIG9yZGVyPTUsIHByaW9yX3NjYWxlPTIpKSwKCiAgICAgICAgICAgIC5zYWZlbHk9RkFMU0UKICAgICAgICApCn0sIG9qX3RyYWlucmVnLCBval9tb2RlbHNldF9iYXNpYykKCm9qX2ZjYXN0X3ByIDwtIHBhcmFsbGVsOjpjbHVzdGVyTWFwKGNsLCBmdW5jdGlvbihtYWJsZSwgbmV3ZGF0YSwgZmNhc3RfZnVuYykKewogICAgbmV3ZGF0YSA8LSBuZXdkYXRhICU+JQogICAgICAgIGZpbGwoZGVhbDptYXhwcmljZWRpZmYsIC5kaXJlY3Rpb249ImRvd251cCIpCiAgICBmY2FzdF9mdW5jKG1hYmxlLCBuZXdkYXRhKQp9LCBval9tb2RlbHNldF9wciwgb2pfdGVzdHJlZywgTW9yZUFyZ3M9bGlzdChmY2FzdF9mdW5jPWdldF9mb3JlY2FzdHMpKQoKZGVzdHJveV9jbHVzdGVyKGNsKQoKc2F2ZV9vYmplY3RzKG9qX21vZGVsc2V0X3ByLCBval9mY2FzdF9wciwKICAgICAgICAgICAgIGV4YW1wbGU9Imdyb2Nlcnlfc2FsZXMiLCBmaWxlPSJtb2RlbF9wci5SZGF0YSIpCgpval9mY2FzdF9wciAlPiUKICAgIGJpbmRfcm93cygpICU+JQogICAgbXV0YXRlX2F0KC0oMTozKSwgZXhwKSAlPiUKICAgIGV2YWxfZm9yZWNhc3RzKCkKYGBgCgpJdCBhcHBlYXJzIHRoYXQgUHJvcGhldCBkb2VzIF9ub3RfIGRvIGJldHRlciB0aGFuIHRoZSBzaW1wbGUgQVJJTUEgbW9kZWwgd2l0aCByZWdyZXNzaW9uIHZhcmlhYmxlcy4gVGhpcyBpcyBwb3NzaWJseSBiZWNhdXNlIHRoZSBkYXRhc2V0IGRvZXMgbm90IGhhdmUgYSBzdHJvbmcgdGltZSBzZXJpZXMgbmF0dXJlOiB0aGVyZSBpcyBubyBzZWFzb25hbGl0eSwgYW5kIG9ubHkgd2VhayBvciBub25leGlzdGVudCB0cmVuZHMuIFRoZXNlIGFyZSBmZWF0dXJlcyB3aGljaCB0aGUgUHJvcGhldCBhbGdvcml0aG0gaXMgZGVzaWduZWQgdG8gZGV0ZWN0LCBhbmQgdGhlaXIgYWJzZW5jZSBtZWFucyB0aGF0IHRoZXJlIHdvdWxkIGJlIGxpdHRsZSBhZHZhbnRhZ2UgaW4gdXNpbmcgaXQuCg==