Recommended way to fit crossvalidated models

[hongooi73]

This touches on fable, fabletools and tsibble so I'm not sure where to put it....

What is the best way to use time series CV to build and test models in the tidyverts framework? Right now, my code is rather manual: I build a sequence of datasets for training and testing, fit models to each training set, and predict on the test set.

subset_oj_data <- function(start, end)
{
    start <- yearweek(start_date + start*7)
    end <- yearweek(start_date + end*7)
    filter(oj_data, week >= start, week <= end)
}

# oj_train is a list of tsibbles
oj_train <- lapply(train_periods,
    function(i) subset_oj_data(settings$FIRST_WEEK, i))

# oj_test is a list of tsibbles
oj_test <- lapply(train_periods,
    function(i) subset_oj_data(i + settings$GAP, i + settings$GAP + settings$HORIZON - 1))

# a list of models, one for each training set
oj_models <- lapply(oj_train,
    function(df) model(df, ar=arima(...))

# a list of forecast results, one for each model/test set
oj_fcasts <- Map(forecast, oj_models, oj_test)

The text at https://otexts.com/fpp3/tscv.html results in null models and error messages when I run it, so it may have been obsoleted since it was written. It also runs forecast with a horizon arg, whereas I'm looking for something to predict on an actual test dataset.

[mitchelloharawild]

The code in fpp3 is not obsolete and is the current recommended approach for cross-validation. Forecasting cross-validated models using new_data requires also folding your new_data to match the .id appropriate for forecasting.

The current method is flexible but cumbersome, and needs a better / alternate interface to do this common task. Unless stretch_tsibble() is dropped from tsibble, then it should remain a viable and flexible method for CV even when new interfaces are designed.

[hongooi73]

Just revisiting this topic: is there a way to use stretch_tsibble to produce a result that doesn't slice the data all the way to the beginning?

That is, say I have a dataset with time periods t=1 to 100. I want to do TS crossvalidation, splitting as follows:

1. training dataset: t=1:50, test dataset: t=51
2. training dataset: t=1:51, test dataset: t=52
3. training dataset: t=1:52, test dataset: t=53
   ... etc ...

Notice that the first 50 periods are always in the training data. Is there a way to get this kind of split with stretch_tsibble?

[mitchelloharawild]

.init=50, .step=1

…

On Fri., 8 May 2020, 08:12 Hong Ooi, ***@***.***> wrote: Just revisiting this topic: is there a way to use stretch_tsibble to produce a result that doesn't slice the data all the way to the beginning? That is, say I have a dataset with time periods t=1 to 100. I want to do TS crossvalidation, splitting as follows: 1. training dataset: t=1:50, test dataset: t=51 2. training dataset: t=1:51, test dataset: t=52 3. training dataset: t=1:52, test dataset: t=53 ... etc ... Notice that the first 50 periods are always in the training data. Is there a way to get this kind of split with stretch_tsibble? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#177 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD3BJF6MGDH7B7FLRKEGIXTRQMW3VANCNFSM4LODPYBA> .