Data storage

[wpbonelli]

Flopy3 simulations have a dictionary in which all model/package data is stored and from which it is looked up. This is somewhat analogous (in concept, not implementation) to the memory manager in mf6. We have discussed whether to keep this approach, or to store data in-place in the component hierarchy in flopy4.

[wpbonelli]

Using xarray as data store means we should be able to easily import/export an entire simulation (or a piece thereof) to a netcdf file. But we need to consider how flopy's internal xarray representation may differ from the netcdf format expected by mf6's new netcdf reader capability, and from the format expected by 3rd party postprocessing tools. It seems ideal if all these representations were to converge, but 3rd party tools will require e.g. extra metadata that we likely don't want to carry around inside flopy. So the ability to map between the representations may be needed.

[wpbonelli]

Copying in some notes from other threads

• per the xarray coordinate docs, x, y, z, time etc are dimensional coords, and nlay, nrow, ncol, kstp, kper etc are non-dimensional coordinates
• we could use custom accessors for e.g. plotting, layers, any information or functionality we want to attach to a dataset
• we've discussed/prototyped a modflow-specific duck array type, supporting e.g. an optional multiplication factor, convenient layer access, and optimized storage for constant arrays — I wonder now if xarray can do at least the first two: layer as a labeled dimension and/or coordinate, factor stored as metadata and accessed/applied at write time
• xarray works with duck arrays so any extra duck array types we provide would work with it

[wpbonelli]

Tentatively we have decided to store data in xarray datasets/trees (depending whether the component is a leaf or node in the component tree). Each component stores its own data in-house, but the datatree provides a unified, hierarchically-addressable view, allowing any component's variables to be accessed from any other component.

Attrs attributes can proxy data stored in xarray data structures, giving us easy introspection and automatic dunder methods without data duplication.

[wpbonelli]

really, this is like an attrs integration for xarray. or vice versa.

which says: give me a bunch of dead simple attrs classes with extra metadata (dims, coords, etc), and I'll tie them into a coherent hierarchical API, with variables aligned along any shared dims, and a datatree backing it.