What is the procedure to get the metrics for a specific variable in a multivariate forecast?

[mreismendes]

Hi! I'd like to know the metrics of a specific variable inside a multivariate forecast. Is it possible?
Thank you!

[winedarksea]

You can, but it is a hidden method and limited.
where model is your fit AutoTS object, run:

model.initial_results.per_series_mae  # if "horizontal" in ensemble
model.initial_results.per_series_spl  # if "probabilistic" in ensemble
model.initial_results.per_timestamp_smape  # if "distance" in ensemble

They are not calculated or saved unless these ensemble methods are enabled in order to reduce memory usage and decrease computation time. You should get an empty DataFrame if you run the above and did not pass the correct ensemble parameter.
To get all of them enabled, at AutoTS init, pass an ensemble parameter like this:

ensemble="simple,distance,horizontal-max,probabilistic-max",

IDs in those tables are the model id (generated by a hash of params) that you can see in the model.results() table.

initial_results will include duplicates (one for each validation). To aggregate:
per_series.groupby(level=0).mean()

[winedarksea]

Also don't forget that you can pass a dictionary of weights to model.fit(...) using weights={"my_variable1":100} to drive model selection towards favoring a variable.
Horizontal-max ensembling is an ensemble where each variable (I normally call a variable a "series") gets the best model for it, run separately. The horizontal ensembling method is semi-hidden at present, while I deliver a few more optimizations to it.