Switch to mosaic for all plotting functionality
#256
Conversation
- Interface to the `plot_horiz_field` has been simplified slightly. - The mask array now must be the same shape as the input field array. - Added a framework function to convert a cell mask to and edge mask.
Also simplified the config options for the colorbar options, which makes the spherical and lat/lon plots have the same options
|
@andrewdnolan, could you change "Fixes #255" to "Partially addresses #255" since we don't want to add python 3.13 testing in this PR. If you leave things as they are, merging this PR will automatically close #255. |
Dependencies for `xarray` based visualization have been removed
TestingSetup for testing: diff --git a/polaris/ocean/tasks/barotropic_gyre/forward.py b/polaris/ocean/tasks/barotropic_gyre/forward.py
index 2424bdc5d..4ab854740 100644
--- a/polaris/ocean/tasks/barotropic_gyre/forward.py
+++ b/polaris/ocean/tasks/barotropic_gyre/forward.py
@@ -139,7 +139,7 @@ class Forward(OceanModelStep):
output_interval_str = time.strftime('0000_%H:%M:%S',
time.gmtime(run_duration))
else:
- stop_time_str = time.strftime('0004-01-01_00:00:00')
+ stop_time_str = time.strftime('0001-04-01_00:00:00')
output_interval_str = time.strftime('0000-01-00_00:00:00')
replacements = dict(You'll also need a config file (below named [barotropic_gyre_default]
steps_to_run = init short_forward long_forward analysisSetting up the baseline: # with the head of E3SM's main branch checked out
source <POLARIS_DIR>/load_dev_polaris_0.5.0-alpha.1_pm-cpu_gnu_mpich.sh
# 9: ocean/planar/barotropic_gyre/default
# 11: ocean/planar/ice_shelf_2d/5km/z-star/default/with_viz
# 65: ocean/spherical/icos/cosine_bell/convergence_space/with_viz
polaris setup -n 9 11 65 -w $PSCRATCH/polaris_PR#256/main/ -f custom.cfg
pushd $PSCRATCH/polaris_PR#256/main/
salloc --nodes 3 --qos interactive --time 01:00:00 --constraint cpu --account m4259
bash job_script.custom.sh
# kill job once it's done Testing this PR's branch: cd <POLARIS_DIR>
git checkout mosaic_viz
source <POLARIS_DIR>/load_dev_polaris_0.5.0-alpha.2_pm-cpu_gnu_mpich.sh
polaris setup -n 9 11 65 -w $PSCRATCH/polaris_PR#256/devel/ -b $PSCRATCH/polaris_PR#256/main/ -f custom.cfg
pushd $PSCRATCH/polaris_PR#256/devel/
salloc --nodes 3 --qos interactive --time 01:00:00 --constraint cpu --account m4259
bash job_script.custom.sh
# kill job once it's done Then, if you want to pull the all of the figures generated by the test suite locally I ran: ssh perlmutter.nersc.gov 'find /pscratch/sd/a/anolan/polaris_PR#256 -name "*.png"' | rsync -vP --files-from=- perlmutter.nersc.gov:/ ~/Desktop/ |
Testing: Spherical ResultsFile to compare: main:
develop: |
Testing: Planar ResultsFile to compare: main: develop: |
Testing: Planer Results (at vertices)File to compare: Note: the error is really large because I only ran this for 3 months, instead of 3 years. main: develop: |
xylar
added
framework
There was a problem hiding this comment.
@andrewdnolan, this looks great!
I do have some suggested changes based just on a code review so far.
Please add mosaic to the intersphinx part of the docs/conf.py file:
https://github.com/andrewdnolan/polaris/blob/mosaic_viz/docs/conf.py#L74
This will allow links to mosaic classes from the documentation (notably the autogenerated API docs).
@cbegeman, can you comment on whether you're comfortable with switching the barotropic gyre plots from contourf to continuous for now? mosaic doesn't (yet?) support contourf.
|
One more thing, we will need to drop python 3.9 support since |
Co-authored-by: Xylar Asay-Davis <xylarstorm@gmail.com>
|
@andrewdnolan, a couple of my suggested renaming changes need other changes to go with them that I didn't try to make. Please add those as well, as things are broken right now. |
@xylar I've got those changes locally, going to remove the |
- Renamed all instances of cell_mask_to_edge_mask and added to API - Fixed typo with adding cartopy features to spherical plots - Added mosaic to intersphinx mapping
|
@andrewdnolan, a quick rebase seems to be needed to fix conflicting updates in After that, I'm happy to run a few tests and then I think I'll be able to approve. |
|
@xylar I still need to update the > git diff --name-only origin/main | grep polaris/ocean/tasks | xargs dirname | sort -u
polaris/ocean/tasks/baroclinic_channel
polaris/ocean/tasks/baroclinic_channel/rpe
polaris/ocean/tasks/barotropic_gyre
polaris/ocean/tasks/cosine_bell
polaris/ocean/tasks/geostrophic
polaris/ocean/tasks/ice_shelf_2d
polaris/ocean/tasks/inertial_gravity_wave
polaris/ocean/tasks/manufactured_solution
polaris/ocean/tasks/sphere_transportAs a final step I'll run a suite with all of those. If you're OK with it, I'm not going to do a baseline like above. I'm just going to run with my branch and do visual inspection that everything looks OK. |
| masked_cells = ds_mesh.nCells.where(~cell_mask, drop=True).astype(int) | ||
|
|
||
| # use inverse so True/False convention matches input cell_mask | ||
| edge_mask = ~cells_on_edge.isin(masked_cells).any("TWO") |
There was a problem hiding this comment.
Does this mean that there has to be unmasked cells on either side of an edge for it to be valid?
There was a problem hiding this comment.
Yes, there does have to be unmasked cells on either side of an edge, but not both, for it to be valid.
With the cells_on_edge array being zero based, boundary edges will have one entry that's >= 0 and one -1 entry. The int array masked_cells is zero based as well, so a value of -1 can never be in it. So, the isin condition will never be true for the missing cell along boundary edges. Therefore, the edge_mask will only ever be true for a boundary edge if the one valid cell on the edge is True in the cell_mask that's passed to the function.
@andrewdnolan Can we switch back to the discrete colorbar here? I chose that intentionally so that the streamlines were more clear. |
|
@andrewdnolan, to get what @cbegeman is asking for, I think it should be possible to just coarsely sample the colormap to get the same effect as |
Discrete colormap is sufficient. Thanks! |
|
There was some offline discussion, but as of 0759678, the barotropic stream function plotting routine uses a discrete colorbar with 21 level for the first two pannel, producing: |
|
@xylar I've updated the |
Testing: full suite of tasks altered by this PRSetup for testing: diff --git a/polaris/ocean/tasks/barotropic_gyre/forward.py b/polaris/ocean/tasks/barotropic_gyre/forward.py
index 2424bdc5d..4ab854740 100644
--- a/polaris/ocean/tasks/barotropic_gyre/forward.py
+++ b/polaris/ocean/tasks/barotropic_gyre/forward.py
@@ -139,7 +139,7 @@ class Forward(OceanModelStep):
output_interval_str = time.strftime('0000_%H:%M:%S',
time.gmtime(run_duration))
else:
- stop_time_str = time.strftime('0004-01-01_00:00:00')
+ stop_time_str = time.strftime('0001-04-01_00:00:00')
output_interval_str = time.strftime('0000-01-00_00:00:00')
replacements = dict(You'll also need a config file (below named custom.cfg) which contains: [barotropic_gyre_default]
steps_to_run = init short_forward long_forward analysis
[inertial_gravity_wave_convergence_space]
steps_to_run = init_200km forward_200km_300s init_100km forward_100km_300s init_50km forward_50km_300s init_25km forward_25km_300s analysis viz
[manufactured_solution_convergence_space]
steps_to_run = init_200km forward_200km_150s init_100km forward_100km_150s init_50km forward_50km_150s init_25km forward_25km_150s analysis vizTesting this PR's branch: # 0: ocean/planar/baroclinic_channel/10km/default
# 4: ocean/planar/baroclinic_channel/10km/rpe
# 9: ocean/planar/barotropic_gyre/default
# 11: ocean/planar/ice_shelf_2d/5km/z-star/default/with_viz
# 28: ocean/planar/inertial_gravity_wave/convergence_space
# 59: ocean/planar/manufactured_solution/convergence_space
# 65: ocean/spherical/icos/cosine_bell/convergence_space/with_viz
# 77: ocean/spherical/icos/geostrophic/convergence_space/with_viz
# 113: ocean/spherical/icos/rotation_2d/with_viz
polaris setup -n 0 4 9 11 28 59 65 77 113 \
-w $PSCRATCH/polaris_PR#256/devel_full_suite/ \
-f custom.cfg
pushd $PSCRATCH/polaris_PR#256/devel_full_suite/
salloc --nodes 3 --qos interactive --time 01:30:00 --constraint cpu --account m4259
bash job_script.custom.sh |
Results : full suite of tasks altered by this PRA bunch of tasks fail, but I don't think for visualization related reasons for any of them.... Results:
Seems to be an issue with running this as an interactive job, maybe? The task is not using the full path to read in the
Again another issue finding the files, not with plotting. Looks like the
The convergence rate is way out of wack here, do I need to be using a specific branch of mpas-ocean to get this to pass? Not totally sure why the |
|
@andrewdnolan, I'll look into the first two failures. Those are bugs, I'm pretty sure.
I think that's expected. We usually test
Yes, that's by design. The |
|
@andrewdnolan It didn't appear as though the images in the visualization documentation page have been changed. I'm particularly curious to see the plotted edge field with edges shown from the baroclinic eddies test case. Thanks! |
|
@cbegeman whoops, I guess I didn't think to update the actual png file. Here's what it looks like: I can add that to the docs. In comparing to the version in the docs, you'll notice the x limits are little more extended in the version above. I haven't explicitly set the axis limits, so matplotlib is just inferring from the plotted data. That's because of the periodic correction of the patches places coordinates outside of the original data limits. I have this flagged in |
Thanks. To me it looks more like some of the kites are ending up on the "wrong" side of the boundary (relative to how I plotted it before). I think it's better to put it on the side with more edge connections. Is that your understanding as well? |
|
@cbegeman My understanding is those dangling kites don't have a "wrong" or "right" side and should actual appear on both sides, but I get that this might look a little different than the previous version of the plot.
I've zoomed in on the region of interest to illustrate what The reason a given periodic patch appears on the left or right side of the plot has to do with the patches center's coordinate (i.e. the associated A long term goal will to be repeat periodic patches on both sides of the boundary, but that's going to require some work since that means adding extra patches (and therefore the length of the patch array will no longer match the length of the data array). I think a shorter term goal will be properly setting the x/y limits for periodic plots, so as use the original coordinate data limits (something like the red lines). |
|
@andrewdnolan, I think until edge kites are plotted on both sides of the periodic boundary, we would still want to keep the whole kite visible rather than cropping half of it out. I think all we want to get the behavior before mosaic would be a "tight" axis that doesn't leave space around the geometry. |
|
Oh, sorry, I see that the old behavior was, in fact, to set the x bounds so that kites were cut off. So my mistake. |
@andrewdnolan, I don't quite follow that argument. It looks to me like both here and in your example in E3SM-Project/mosaic#23 (comment), there are clearly some |
|
@xylar Thanks for fixing those issues! I'm going to rebase locally and then try to re-run the suite. I'll switch manufactured solution run to be both space and time. I've been doing all this testing on perlmutter so I'm going to try and put a bow on my testing by the end of the day today, with |
|
@andrewdnolan, It seems to me the issue is with the edge whose center I've marked with an X and the missing periodic image I've marked with an O. It's visually clear that that edge is in the wrong place, at least to me.
|
|
I believe where you have drawn the periodic second boundary in #256 (comment) is off by about a half an edge kite from where it should be. Was that drawn by hand or is that something computed in mosaic? |
@xylar I agree the right boundary is off by half an edge kite in the figure I created. (Yes, those where drawn manually using the min/max of I agree that the |
|
I think it should be min(xEdge) to min(xEdge)+xPeriod. |
|
That makes sense to me. I'll open PR in |
Partially addresses #255 and fixes the plotting issues from #242.
Checklist
api.md) has any new or modified class, method and/or functions listedTestingcomment in the PR documents testing used to verify the changes