Skip to content

Comments

Add suggestions for xarray .open_mfdataset parameters#215

Merged
grantbuster merged 13 commits intomainfrom
pp/xr_mf_suggestions
May 19, 2025
Merged

Add suggestions for xarray .open_mfdataset parameters#215
grantbuster merged 13 commits intomainfrom
pp/xr_mf_suggestions

Conversation

@ppinchuk
Copy link
Collaborator

By default xarray will attempt to validate coordinates during concatenation. Since rex-style data coordinates are known to match, it;s better to skip this validation step in order to get a huge performance gain when using xarray .open_mfdataset. This PR adds some notes about this and a warning to the open_mfdataset_hsfs function

@ppinchuk ppinchuk requested a review from grantbuster May 19, 2025 00:52
@ppinchuk ppinchuk self-assigned this May 19, 2025
@ppinchuk ppinchuk added documentation Improvements or additions to documentation enhancement Update to logic or general code improvements labels May 19, 2025
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 67.70%. Comparing base (74e8acf) to head (1fc05e8).

Files with missing lines Patch % Lines
rex/external/rexarray.py 0.00% 3 Missing ⚠️

❌ Your patch status has failed because the patch coverage (0.00%) is below the target coverage (70.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #215      +/-   ##
==========================================
- Coverage   67.73%   67.70%   -0.03%     
==========================================
  Files          40       40              
  Lines        7095     7098       +3     
  Branches     1039     1040       +1     
==========================================
  Hits         4806     4806              
- Misses       1961     1964       +3     
  Partials      328      328              
Flag Coverage Δ
unittests 67.70% <0.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
rex/external/rexarray.py 73.52% <0.00%> (-0.58%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@grantbuster grantbuster merged commit 99ba31a into main May 19, 2025
19 checks passed
@grantbuster grantbuster deleted the pp/xr_mf_suggestions branch May 19, 2025 14:55
github-actions bot pushed a commit that referenced this pull request May 19, 2025
Add suggestions for `xarray .open_mfdataset` parameters
github-actions bot pushed a commit to MRE-Code-Hub/rex that referenced this pull request May 19, 2025
Add suggestions for `xarray .open_mfdataset` parameters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement Update to logic or general code improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants