Turning on diagnostic outputs in FV3GFS

[mdtoy]

I'm trying to turn on diagnostic outputs in the FV3GFS/CCPP and I’m finding it’s not a trivial process. It failed with the “ugwpv1” suite, so I tried for simplicity by testing with the vanilla “FV3_GFS_v16” suite, and it also fails there. Is there any documentation/guidance on how to do this?
What I did was to set “lssav” and “ldiag3d” equal to .true. and the model crashes right away (I’m looking into why). My goal is to output and analyze the vertical momentum fluxes from each of the drag components of the ugwpv1 scheme which Valery Yudin set up. This will guide me on how to tune each one up.

[yangfanglin]

@junwang-noaa @SMoorthi-emc Jun and Moorthi, any tips you can share with Mike ? thanks

[climbfuji]

If this is related to CCPP tendencies, there is a fair amount of documentation in the CCPP technical documentation, https://ccpp-techdoc.readthedocs.io/en/latest/ - section 10.2.

[mdtoy]

Thanks @climbfuji, this documentation is just what I was looking for.

[mdtoy]

@climbfuji When we coded the "unified_ugwp" and "ugwpv1_gsldrag" CCPP schemes, we provided extra diagnostic fields for tendencies specific to each component of the GSL drag suite, e.g., dudt_ofd (tendency_of_x_momentum_due_to_form_drag), du_ofdcol (vertically_integrated_x_momentum_flux_due_to_form_drag), etc. I'm following Section 10 of the CCPP tech document you directed me to. Would these diagnostic fields be considered "Auxiliary Arrays", and would section 10.3 pertain to having these fields be output?

[climbfuji]

These arrays still exist and are part of the Diag GFS DDT. I am not sure all of them (simply too many) were added to GFS_diagnostics.F90. You could just check in that file. If they are, then you only need to add the corresponding entry in your diag_table to get the data written out.

[mdtoy]

So, if they're not in GFS_diagnostics.F90 I just need to add them. And since they're already a part of the Diag GFS DDT I don't need to deal with auxiliary diagnostics, i.e., aux3d, naux3d, etc. Is this correct?

[climbfuji]

Correct.

[mdtoy]

Hi @everyone I found the problem, at least for the "FV3_GFS_v16" case. The solution to getting past the crash is turning on the flag "qdiag3d = .true."
The issue is a mismatch between the flagging conditions for allocating certain diagnostic variables in GFS_typedefs.F90 versus when they're used in the relevant CCPP module. To be specific, in the case I encountered, the variable "upd_mf" is only allocated in GFS_typedefs if "qdiag3d=.true.", but this condition isn't imposed on line 173 of GFS_DCNV_generic.F90 where it is set. This is the line where the crash occurred.
As a general comment, there seem to be multiple instances of logic-switch mismatches between variables being allocated in GFS_typedefs and then being used by the various modules. I guess we should fix them as they're encountered.

[climbfuji]

Good finding, Mike. Yes, please, whenever you encounter such mismatches please record them as an issue in ccpp-physics and ideally create a PR to fix them. The issue is the most important part, however.

Note also that we are currently moving towards using assumed-size array declarations for all array arguments, i.e. use

real(kind_phys), dimension(:,:), intent(inout) :: foo

instead of

real(kind_phys), dimension(im,levs), intent(inout) :: foo

This allows the compiler to do bounds checking, which will greatly facilitate detecting the reasons for segmentation faults. There is a draft PR (work in progress) with a lot of changes here, if you want to take a look: #611

[SMoorthi-emc]

Fanglin, I have never run with diagnostics turned on in this model. I don't know if it works. Moorthi

…

On Fri, Apr 16, 2021 at 5:14 PM Fanglin Yang ***@***.***> wrote: @junwang-noaa <https://github.com/junwang-noaa> @SMoorthi-emc <https://github.com/SMoorthi-emc> Jun and Moorthi, any tips you can share with Mike ? thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#631 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALLVRYXIXNT3BEDZ6HGP2QDTJCSD7ANCNFSM43CFY5WA> .

-- Dr. Shrinivas Moorthi Research Meteorologist Modeling and Data Assimilation Branch Environmental Modeling Center / National Centers for Environmental Prediction 5830 University Research Court - (W/NP23), College Park MD 20740 USA Tel: (301)683-3718 e-mail: ***@***.*** Phone: (301) 683-3718 Fax: (301) 683-3718

[ericaligo-NOAA]

Several months ago I did write out diagnostics, and I only set ldiag3d = .true.  lssav is not in my input.ml file, but by default it was set to false. I had added these to my diag_table: < "gfs_phys",    "dt3dt_mp",     "dt3dt_mp", "fv3_history2d",  "all",  .false.,  "none",  2 < "gfs_phys",    "dv3dt_mp",     "dv3dt_mp", "fv3_history2d",  "all",  .false.,  "none",  2 < "gfs_phys",    "dq3dt_mp",     "dq3dt_mp", "fv3_history2d",  "all",  .false.,  "none",  2 Sorry if this was not helpful. Eric

…

On 4/16/2021 4:26 PM, mdtoy wrote: I'm trying to turn on diagnostic outputs in the FV3GFS/CCPP and I’m finding it’s not a trivial process. It failed with the “ugwpv1” suite, so I tried for simplicity by testing with the vanilla “FV3_GFS_v16” suite, and it also fails there. Is there any documentation/guidance on how to do this? What I did was to set “lssav” and “ldiag3d” equal to .true. and the model crashes right away (I’m looking into why). My goal is to output and analyze the vertical momentum fluxes from each of the drag components of the ugwpv1 scheme which Valery Yudin set up. This will guide me on how to tune each one up. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#631>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALQ75MPS3F34K24J5TOO6JTTJCMNVANCNFSM43CFY5WA>.

[mdtoy]

I issued a PR with the bug fixes. Should I record the problem in "Issues" anyway?