Training one model per unique_id instead of horizon ?

[mat-ej]

Hi Nixtla team, thanks for wonderful library.

Can I kindly ask you for help with the following:

Assume I have a very typical prediction dataframe where id_col column identifies how "far" the target_date is from run_date of the model prediction. This identifies what kind of covariates are available( or better how accurate they are according to when I am running the prediction)

 ─────────────────────┬─────────────────────┬────────┬────────────┬────────────┐
│ target_date         ┆ run_date            ┆ id_col ┆ cov1       ┆ cov2       │
│ ---                 ┆ ---                 ┆ ---    ┆ ---        ┆ ---        │
│ datetime[ns]        ┆ datetime[ns]        ┆ i16    ┆ f64        ┆ f64        │
╞═════════════════════╪═════════════════════╪════════╪════════════╪════════════╡
│ 2021-01-01 06:30:00 ┆ 2021-01-01 05:45:00 ┆ 3      ┆ 0.0        ┆ 0.0        │
│ 2021-01-01 06:45:00 ┆ 2021-01-01 05:45:00 ┆ 4      ┆ 0.0        ┆ 0.0        │
│ 2021-01-01 06:45:00 ┆ 2021-01-01 06:00:00 ┆ 3      ┆ 0.0        ┆ 0.0        │
│ 2021-01-01 06:45:00 ┆ 2021-01-01 06:30:00 ┆ 1      ┆ 3.622404   ┆ 3.088123   │

(Effectively the dataset is extended by the fcst horizon for each timestep.)

Initially I thought I could simply give each of these a unique id_col and I would be able to train a model per id_col but that does not seem to be the case.

I also do not see how training a model per horizon would work in this case as I will have multiple identical target_dates but with different covariates according to run_date. The time index wont be complete in either case for some target_dates only some run_dates etc, as shown in the example.

(I know this is a lot more of a typical clasif/regr view on the problem instead of timeseries, but I do like the Nixtla ecosystem of distributed training, multithreading, preprocessing and error metrics so I would love to use mlforecast)

How can this be done in the best way possible? Can I somehow train model per id_col ? (Or am I perhaps not thinking about this correctly, can I somehow tell the model what the run_date is in the dataframe ?)

I will be grateful for any help or guidance.

Have a great day. : )

[jmoralez]

Hey. mlforecast trains a single global model, so if you want to have one model per id you'd need to have one MLForecast instance per id.

I'm struggling to see the difference with the one model per horizon approach here, since that's pretty much what it does (train one model for one step ahead, one for two steps ahead, etc). We don't have a run_date but it's kind of implicit, since if you have daily data and the value of your time_col (run_date) is 2000-01-01 then that row will have the values for all your covariates at that time and if you set max_horizon=3 you'll have targets associated to (target_date) 2000-01-01, 2000-01-02 and 2000-01-03. You can experiment with this by running MLForecast.preprocess and setting the max_horizon argument.

For the missing timestamps you could use fill_gaps.

[mat-ej]

Yes, the only difference is that I do have multiple data sources that are delayed in distinct ways.
-The ground truths are delayed in a distinct way and the covariates are delayed in distinct ways.

1. I could align all to a single time of the ground truths using some merge asof approach.

2. Or I could train model per (ground truth delay, covs delay)

In the second option this would allow me to analyze all possible "situations" and their effect on metrics etc. (By including all "covariate shifts" in my dataframe I would also extend the dataset dramatically)

Thanks for answer, I ll check fill_gaps method and preprocess.

[jmoralez]

Hey. We recently added support for scikit-learn models in statsforecast (which trains one model per id). It isn't released yet but if you can install from github you can try it out.