2nd AtmoRep roadmap meeting

07-10-2024 2nd AtmoRep roadmap meeting

General

• Start: 14:20, end: 15:10
• Participants: Christian Lessig, Martin Schultz, Michael Langguth, Enxhi Krespha, Siddhant Agarwal, Nishant Kumar, Ilaria Luise, David Greenberg, Kacper Nowak, Asma Semcheddine, Michael Tarnawa

Meeting notes

• datasets
  • OceanRep status
    • training data is prepared at AWI (zarr-output), models are currently training
    • first coupled run expected in the next weeks (FESOM with ERA5 forcing)
      • running at 1°deg, daily output
    • architetcure/model configuration
      • working at DKRZ
      • 8 nodes for training
      • 12 layers in encoder and decoder (by contrast to more light-weight current config with six layers)
      • Kacper will share his specific config with Christian and Ilaria
      • issues with memory consumption and masking
  • other: total precipitation data is available in sfc_data in zarr
• downscaling
  • current status:
    • initial branch set-up, see here
    • first integration of PerceiverIO-module and simple downscaling network
    • current focus: get training to work
      • prepare IMERG precipitation data
      • adapt/use dataset sampler to read in IMERG data for training
  • other notes
    • in AIFS: no added value from high-resolved topography -> focus on getting training to run, not ingesting additional data
    • partition between convective and large-scale precipitation beneficial? -> ask Paula
    • test if output from intermediate layers is beneficial (down to encoder)
    • first Multiformer prototype excl. total precipitation will be available by end of this week
    • follow-up Multiformer incl. total precipitation one/two weeks later
    • consider training/working on BSC (I have access)
• forecasting/ roll-out
  • Christian is implementing on forecasting (deterministic) performed in latent space
    • first results by end of this week -> branch on github
    • further clean-up afterwards
    • get latent representation from all patches and merge into global latent representation
    • do forecasting with global latent representation
    • latent space representation includes at least three hours of data, but can be more
    • can probably trained end-to-end (incl. core model) with the lightweight model architecture
  • Nishant is working on forecasting with diffusion model, see this issue-branch
    • Christian suggest to condition diffusion model with data from latent space, i.e. after encoder
    • ideally: have a pre-trained, multi-purpose encoder-decoder and put a forecast engine (e.g. a diffusion model) in between working with latent representation -> would fit the concept of a Foundation Model
    • more details to be discussed in a personal talk between David, Nishant, Christian and Ilaria
• Hyperparameter tuning (work by Nishant)
  • paper on Tensor Programs V by Microsoft that could be implemented along with AtmoRep
  • is orthogonal to other work
  • be carful with resource usage, but is labelled as not requiring excessive resources
• JUREAP
  • not on the lightening track -> AtmoRep is not highly scalable
  • some scaling experiments required as an entry card to JUPITER
  • benchmarks required, see issue-branch #32
    • benchmark on the inference and training
    • use Multiformer/Singleformer trained by Ilaria
• NIC Symposium paper
  • JSC have been invited to submit a short paper
  • draft with focus on Asma's data compression work and downscaling under preparation
  • update discussion internally at JSC this Wednesday
• status update on parallel processing of AtmoRep output
  • will also be discussed internally on Wednesday