-
Notifications
You must be signed in to change notification settings - Fork 175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates for cycling C1152 ATM #3173
Comments
@WenMeng-NOAA : Would you be able to take a look at the atmanl_upp failure? I copied the log file onto Hera here: |
@CatherineThomas-NOAA I will look into this issue. |
@CatherineThomas-NOAA From your runtime log:
I would suggest increasing the total tasks from 120 to 200 or more and reducing tasks per node to allocate more memory for C1152. My own UPP standalone test for C1152 on WCOSS2 is configurated as: mpiexec -l -ppn 40 -n 240 |
@WenMeng-NOAA: Thanks for the configuration suggestion. I'll try it once WCOSS2 is back. |
Description updated to replace NOAA-EMC/GSI-utils#57 with NOAA-EMC/GSI-utils#59. Testing of NOAA-EMC/GSI-utils#59 is on hold until WCOSS2 returns. |
A handful of updates are needed to cycle with C1152 atmosphere, which are mostly related to configs and resources. Refs: NOAA-EMC#3173
Config changes have been included in my fork here. These config changes were added by hand based on previous WCOSS testing. This particular branch has only been tested on Hera to make sure it didn't fail when creating the XML. Once WCOSS returns, it can be tested properly with C1152 including the updates to GSI-utils and upp node configuration. |
I reran my C1152 ATM-only cycling experiment with my fork and changes from NOAA-EMC/GSI-utils#59. The analcalc and atmanl upp jobs completed without error. As a sanity check, I plotted the gsistats over a few cycles and results were consistent with operations, with some notable improvements: I had an error in the archive step which was related to obs, but I jerry rigged the DMPDIR for this case, so it's likely this is not a typical error. Unfortunately, my logs have scrubbed so I can't follow up. |
What is wrong?
This issue is for tracking the updates needed to cycle C1152 atmosphere-only. Changes found so far in testing:
What should have happened?
C1152 ATM-only should cycle without failure on WCOSS2.
What machines are impacted?
All or N/A, WCOSS2
What global-workflow hash are you using?
bc61862
Steps to reproduce
Additional information
Logs and expdirs have been copied on to Hera:
/scratch1/NCEPDEV/da/Catherine.Thomas/v17/c1152
Do you have a proposed solution?
Each problem and potential solutions are being documented in this issue as they arise. Once C1152/C384 ATM-only can run without error on WCOSS2, I will open a PR with the needed changes.
The text was updated successfully, but these errors were encountered: