[929][evalution/io] Make lead time and source windew available to evaluation via OutputDataset

[grassesi]

Description

Store source window as a attribute of the zarr group corresponding to one OuputDataset. When converting OutputDataset to xarray, source window is available as a coordinate over the axis "sample". Lead time can be calculated from the source window and the absolute time.

Issue Number

closes #929

Is this PR a draft? Mark it as draft.

Checklist before asking for review

• I have performed a self-review of my code
• My changes comply with basic sanity checks:
  • I have fixed formatting issues with ./scripts/actions.sh lint
  • I have run unit tests with ./scripts/actions.sh unit-test
  • I have documented my code and I have updated the docstrings.
  • I have added unit tests, if relevant
• I have tried my changes with data and code:
  • I have run the integration tests with ./scripts/actions.sh integration-test
  • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
  • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
• I have informed and aligned with people impacted by my change:
  • for config changes: the MatterMost channels and/or a design doc
  • for changes of dependencies: the MatterMost software development channel

[tjhunter]

@grassesi a few comments. Something does not seem right in one piece.

Also, your code is not backward compatible. @iluise : is this fine for you that all the sameples are invalidated? We can make it backward compatible but I am not sure it is worth it.

[tjhunter]

are you using it? I don't see a place where you would need it.

Also, let's not used cached_property unless it is vital to do so. Most people will be confused.

[grassesi]

@grassesi a few comments. Something does not seem right in one piece.
Yes it is very much untested, I need to do that today, just wanted to get it out there.

Also, your code is not backward compatible. @iluise : is this fine for you that all the sameples are invalidated? We can make it backward compatible but I am not sure it is worth it.
I would very much avoid implementing Backward compatibility for this one 😅

[iluise]

@grassesi a few comments. Something does not seem right in one piece.

Also, your code is not backward compatible. @iluise : is this fine for you that all the sameples are invalidated? We can make it backward compatible but I am not sure it is worth it.

Yes, no problem for the backward compatibility. If we have the checkpoints we can always re-run inference so I'd not worry about it.

[iluise]

Hi Simon,

I tested the PR and we are close. Small requests:

• can we remove the 0 from the zio.forecast_steps when it is not predicted? If not we will always get forecast_steps = [0-x] even if we don't have target or prediction in the zarr for setp 0.

• I can't retrieve source from forecast step 0. I get the following:

out = zio.get_data('0', stream, '0')
*** ValueError: Missing target dataset for item: 0/ERA5/0

is there a better way to retrieve source directly without passing through zio.get_data('0', stream, '0')?