Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor storage #133

Merged
merged 8 commits into from
May 13, 2021
Merged

Refactor storage #133

merged 8 commits into from
May 13, 2021

Conversation

rabernat
Copy link
Contributor

The goal of this refactor is to remove code from recipes/xarray_zarr.py and add code to storage.py. The aim of this is to improve our abstractions around file handling, such that most of the generic logic related to downloading, caching, and copying files lives in the storage module.

The main new additions are:

  • A context manager function called storage.file_opener which takes care of downloading, caching, and copying to local tmpdir
  • A new MetadataTarget class which serializes / deserializes data from / to python dictionaries. (We were already doing this in xarray_zarr.py but now it lives in storage
  • Better type hints, which caught some edge cases
  • Removal of the UninitializedTarget class. I'm sure there was some reason for this at some point, but it seems unnecessary now.

@rabernat rabernat requested a review from cisaacstern May 12, 2021 18:46
Copy link
Member

@cisaacstern cisaacstern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few questions in-line. Some of the internals are still a bit esoteric to my eye, but reading through this was a good chance to take another step towards understanding the inner workings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants