Update indexer to treat .zarr as files not dirs #1046
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Microscopy file formats were recently updated in the standard to include ome-zarr, which are stored in a directory suffixed with
.ome.zarr
.https://bids-specification.readthedocs.io/en/stable/modality-specific-files/microscopy.html#file-formats
This however doesn't work with pybids currently, since the pybids just indexes it as a directory.. This can cause issues if there is an ome.zarr folder with real data, since there are a large number of files in that folder which slow the indexing to a crawl..
This adds a quick and simple (albeit hardcoded) fix to ensure .zarr directories get indexed as files instead of directories. This enables us to get .ome.zarr microscopy files, and also prevents needless indexing of .zarr directories that contain a large number of files.
This solves my problem as I can now use ome.zarr in bids, seemingly without any negative side effects (?) - a more configurable solution would seem nicer of course, but not sure how often this would come up in the future (ie bids "files" that are actually directories), so not sure it would be worth the added config set-up?
Thoughts?
Pinging @satra as someone in the BIDS community that was also involved in the ome-zarr standard.
Note: .ome.zarr does not yet seem to be in the pybids config, so using validate=False in the layout to test things..