Issues with GFDL Model Output Variable Attributes

[jtmims]

While working on the framework and PODs from previous phases, a few quirks with the attribute metadata in GFDL output have been noticed by @wrongkindofdoctor and me. This issue will serve as a discussion board for the the GFDL MDTF community and framework devs to discuss irregularities and come to solutions. This will be continually updated as new issues arise. Please comment any found.

Some weirdness found:

• The units for siconc is 0-1 when it has historically been %

[wrongkindofdoctor]

@bitterbark Suggested forcing variable metadata that is present but does not match expected values to standard definitions in the field list/CV tables, and providing detailed logs of the changes. Some metadata, such as units, could be excluded from the initial catalog query and updated as needed during preprocessing. However, information like standard_name, realm, and grid type that are required to yield unique query results need cleaning before performing the query. Ideally, QC on the model output files would include this step, but defining and enforcing metadata procedures is outside the scope of the MDTF. Instead, missing/disparate metadata could be replaced with information from the corresponding field tables during catalog generation/updating. It might be beneficial to include a column for notes on metadata that differs from the file contents when these changes occur.

[bitterbark]

The more we can allow the MDTF to run on non-standard files, the more CESM developers will actually use it. The more restrictive we make it, the harder it will be to use and therefore the less it will be used by CESM. It should not be the MDTF's job to specify anything beyond what is *needed* to run the MDTF. For instance, when searching the catalog, if the variable_id and the frequency are found, the only reason to check units is to perform a known unit conversion. If the units are not standard, a courtesy warning can be issued and then the MDTF should try to proceed. We should not protect users from themselves at the cost of making the package more difficult to run. If it isn't obvious that users should use their best judgement interpreting the results, we could state that explicitly.

…

On Thu, Dec 19, 2024 at 10:24 AM Jess ***@***.***> wrote: @bitterbark <https://github.com/bitterbark> Suggested forcing variable metadata that is present but does not match expected values to standard definitions in the field list/CV tables, and providing detailed logs of the changes. Some metadata, such as units, could be excluded from the initial catalog query and updated as needed during preprocessing. However, information like standard_name, realm, and grid type that are required to yield unique query results need cleaning before performing the query. Ideally, QC on the model output files would include this step, but defining and enforcing metadata procedures is outside the scope of the MDTF. Instead, missing/disparate metadata could be replaced with information from the corresponding field tables during catalog generation/updating. It might be beneficial to include a column for notes on metadata that differs from the file contents when these changes occur. — Reply to this email directly, view it on GitHub <#723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHLJVQFLIO3KBSPSSCPJ5LL2GL6MPAVCNFSM6AAAAABTZHMRHKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJVGI2TAOBRGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- My working day may not be the same as your working day. Please do not feel obliged to reply to this email outside of your normal hours.