Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for tracer to not be advected by SHOC #3085

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions components/eamxx/src/control/atmosphere_driver.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -534,6 +534,52 @@ void AtmosphereDriver::create_fields()
m_field_mgrs[grid->name()]->registration_begins();
}

// Before registering fields, check that Field Requests for tracers are compatible
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this snippet should be moved somewhere, like in share/util, or maybe as a free function in field_request.hpp, something like

// ensure all requests are on the same page on whether the field belongs to $group_name
bool compatible_groups (std::vector<FieldRequest>& reqs, std::string& group_name);

(maybe with a better name)

// Create map from tracer name to a vector which contains the field requests for that tracer.
std::map<std::string, std::set<FieldRequest>> tracer_requests;
auto gather_tracer_requests = [&] (FieldRequest req) {
if (not ekat::contains(req.groups, "tracers")) return;

std::string fname = req.fid.name();
if (tracer_requests.find(fname) == tracer_requests.end()) {
tracer_requests[fname] = {req};
} else {
tracer_requests[fname].emplace(req);
}
};
for (const auto& req : m_atm_process_group->get_required_field_requests()){
gather_tracer_requests(req);
}
for (const auto& req : m_atm_process_group->get_computed_field_requests()) {
gather_tracer_requests(req);
}

// Go through the map entry for each tracer and check that every one
// has the same request for turbulence advection.
for (auto fr : tracer_requests) {
const auto reqs = fr.second;

std::set<bool> turb_advect_types;
for (auto req : reqs) {
turb_advect_types.emplace(ekat::contains(req.groups, "turbulence_advected_tracers"));
}

if (turb_advect_types.size()!=1) {
std::ostringstream ss;
ss << "Error! Incompatible tracer request. Turbulence advection requests not consistent among processes.\n"
" - Tracer name: " + fr.first + "\n"
" - Requests (process name, grid name, is tracers turbulence advected):\n";
for (auto req : reqs) {
const auto grid_name = req.fid.get_grid_name();
const bool turb_advect = ekat::contains(req.groups, "turbulence_advected_tracers");
ss << " - (" + req.calling_process + ", " + grid_name + ", " + (turb_advect ? "true" : "false") + ")\n";
}
EKAT_ERROR_MSG(ss.str());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sample output in case that P3 requests qv to not be advected by SHOC.

Error! Incompatible tracer request. Turbulence advection requests not
  consistent among processes.
    - Tracer name: qv
    - Requests (process name, grid name, is tracers turbulence advected):
      - (homme, Physics GLL, true)
      - (mam4_aci, Physics GLL, true)
      - (p3, Physics GLL, false)
      - (homme, Physics GLL, true)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there a repeated entry for homme?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is because there exists a computed and required group request (since "qv" is "updated" in homme), but then I don't know why p3 does not have 2 requests as well. Let me investigate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So yes, double entries are due to "updated" fields, since a request exists for "required" and "computed".

Also, field requests in ATMProcGroup is a set, and so only one version of each reqest is used. By chance homme's "qv" was the version of "qv" with packsize>1, P3 was picked up since it's req.groups was unique (i.e., no turbulence advection), and mam4_aci was picked up as the packsize=1 request (all other mam "qv" request ignored.

@bartgol I've pushed a solution that does the following

  1. Makes tracer_requests a map of set of field requests, so removes duplicate computed/required entry.
  2. Use req.calling_process in FieldRequest::operator< so that requests with different calling_process are distinct (when loaded into a std::set<FieldRequest>). The m_*_field_requests in AtmosphereProcess is now larger, but it does not result in more allocations since FM will skip extra requests when it allocates.

The other options for 2. I think are
a. remove all calling process info in FieldRequest and simplify the error message with no calling process and just say "qv has incompatible group requests" and make the user find the processes,
b. or leave as it was in the output above which is not complete but gives user the name of conflicting processes.

Copy link
Contributor

@bartgol bartgol Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could put an enum in the request, that specifies whether this req is for input, output, or input-output. It may be helpful for debugging purposes.

That said, I don't think that seeing duplicated processes in the output is a big deal. What I would do, maybe, is explain why the requests are incompatible. That is, you could add a line at the bottom of the error msg, like

Error! Incompatible tracer request. Turbulence advection requests not
  consistent among processes.
    - Tracer name: qv
    - Requests (process name, grid name, is tracers turbulence advected):
      - (homme, Physics GLL, true)
      - (mam4_aci, Physics GLL, true)
      - (p3, Physics GLL, false)
      - (homme, Physics GLL, true)
  All processes MUST agree on whether this tracer is advected by the turbulence scheme.

}
}
}

// Register required/computed fields
for (const auto& req : m_atm_process_group->get_required_field_requests()) {
m_field_mgrs.at(req.fid.get_grid_name())->register_field(req);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ void SurfaceCouplingExporter::set_grids(const std::shared_ptr<const GridsManager
add_field<Required>("phis", scalar2d_layout, m2/s2, grid_name);
add_field<Required>("p_mid", scalar3d_layout_mid, Pa, grid_name, ps);
add_field<Required>("T_mid", scalar3d_layout_mid, K, grid_name, ps);
add_tracer<Required>("qv", m_grid, kg/kg, ps);
add_tracer<Required>("qv", m_grid, kg/kg, true, ps);
// TODO: Switch horiz_winds to using U and V, note right now there is an issue with when the subfields are created, so can't switch yet.
add_field<Required>("horiz_winds", vector3d_layout, m/s, grid_name);
add_field<Required>("sfc_flux_dir_nir", scalar2d_layout, W/m2, grid_name);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ void HommeDynamics::set_grids (const std::shared_ptr<const GridsManager> grids_m
add_field<Computed>("p_dry_mid", pg_scalar3d_mid, Pa, pgn,N);
add_field<Computed>("omega", pg_scalar3d_mid, Pa/s, pgn,N);

add_tracer<Updated >("qv", m_phys_grid, kg/kg, N);
add_tracer<Updated >("qv", m_phys_grid, kg/kg, true, N);
add_group<Updated>("tracers",pgn,N, Bundling::Required);

if (fv_phys_active()) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ void CldFraction::set_grids(const std::shared_ptr<const GridsManager> grids_mana

// Set of fields used strictly as input
constexpr int ps = Pack::n;
add_tracer<Required>("qi", m_grid, kg/kg, ps);
add_tracer<Required>("qi", m_grid, kg/kg, true, ps);
add_field<Required>("cldfrac_liq", scalar3d_layout_mid, nondim, grid_name,ps);

// Set of fields used strictly as output
Expand Down
6 changes: 3 additions & 3 deletions components/eamxx/src/physics/cosp/eamxx_cosp.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,9 @@ void Cosp::set_grids(const std::shared_ptr<const GridsManager> grids_manager)
add_field<Required>("phis", scalar2d , m2/s2, grid_name);
add_field<Required>("pseudo_density", scalar3d_mid, Pa, grid_name);
add_field<Required>("cldfrac_rad", scalar3d_mid, nondim, grid_name);
add_tracer<Required>("qv", m_grid, kg/kg);
add_tracer<Required>("qc", m_grid, kg/kg);
add_tracer<Required>("qi", m_grid, kg/kg);
add_tracer<Required>("qv", m_grid, kg/kg, true);
add_tracer<Required>("qc", m_grid, kg/kg, true);
add_tracer<Required>("qi", m_grid, kg/kg, true);
// Optical properties, should be computed in radiation interface
add_field<Required>("dtau067", scalar3d_mid, nondim, grid_name); // 0.67 micron optical depth
add_field<Required>("dtau105", scalar3d_mid, nondim, grid_name); // 10.5 micron optical depth
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,19 +82,19 @@ void MAMAci::set_grids(

// atmospheric quantities
// specific humidity [kg/kg]
add_tracer<Required>("qv", grid_, q_unit);
add_tracer<Required>("qv", grid_, q_unit, true);

// cloud liquid mass mixing ratio [kg/kg]
add_tracer<Required>("qc", grid_, q_unit);
add_tracer<Required>("qc", grid_, q_unit, true);

// cloud ice mass mixing ratio [kg/kg]
add_tracer<Required>("qi", grid_, q_unit);
add_tracer<Required>("qi", grid_, q_unit, true);

// cloud liquid number mixing ratio [1/kg]
add_tracer<Required>("nc", grid_, n_unit);
add_tracer<Required>("nc", grid_, n_unit, true);

// cloud ice number mixing ratio [1/kg]
add_tracer<Required>("ni", grid_, n_unit);
add_tracer<Required>("ni", grid_, n_unit, true);

// Temperature[K] at midpoints
add_field<Required>("T_mid", scalar3d_mid, K, grid_name);
Expand Down Expand Up @@ -157,15 +157,16 @@ void MAMAci::set_grids(

// interstitial and cloudborne aerosol tracers of interest: mass (q) and
// number (n) mixing ratios
// NOTE:
// - For interstitial aerosols, we have dynamics advect, but not turbulence.
// - For cloudborne aerosols, DO NOT advect.
bartgol marked this conversation as resolved.
Show resolved Hide resolved
for(int mode = 0; mode < mam_coupling::num_aero_modes(); ++mode) {
// interstitial aerosol tracers of interest: number (n) mixing ratios
const char *int_nmr_field_name =
mam_coupling::int_aero_nmr_field_name(mode);
add_tracer<Updated>(int_nmr_field_name, grid_, n_unit);
add_tracer<Updated>(int_nmr_field_name, grid_, n_unit, false);

// cloudborne aerosol tracers of interest: number (n) mixing ratios
// NOTE: DO NOT add cld borne aerosols to the "tracer" group as these are
// NOT advected
const char *cld_nmr_field_name =
mam_coupling::cld_aero_nmr_field_name(mode);
add_field<Updated>(cld_nmr_field_name, scalar3d_mid, n_unit, grid_name);
Expand All @@ -175,11 +176,9 @@ void MAMAci::set_grids(
const char *int_mmr_field_name =
mam_coupling::int_aero_mmr_field_name(mode, a);
if(strlen(int_mmr_field_name) > 0) {
add_tracer<Updated>(int_mmr_field_name, grid_, q_unit);
add_tracer<Updated>(int_mmr_field_name, grid_, q_unit, false);
}
// (cloudborne) aerosol tracers of interest: mass (q) mixing ratios
// NOTE: DO NOT add cld borne aerosols to the "tracer" group as these are
// NOT advected
const char *cld_mmr_field_name =
mam_coupling::cld_aero_mmr_field_name(mode, a);
if(strlen(cld_mmr_field_name) > 0) {
Expand All @@ -190,7 +189,7 @@ void MAMAci::set_grids(

for(int g = 0; g < mam_coupling::num_aero_gases(); ++g) {
const char *gas_mmr_field_name = mam_coupling::gas_mmr_field_name(g);
add_tracer<Updated>(gas_mmr_field_name, grid_, q_unit);
add_tracer<Updated>(gas_mmr_field_name, grid_, q_unit, true);
} // end for loop num gases

// ------------------------------------------------------------------------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,7 @@ class MAMAci final : public scream::AtmosphereProcess {
// for atmosphere
compute_vertical_layer_heights(team, dry_atm_pre_, i);
compute_updraft_velocities(team, wet_atm_pre_, dry_atm_pre_, i);
set_min_background_mmr(team, dry_aero_pre_, i); //dry_atm_pre_ is the output
} // operator()

// local variables for preprocess struct
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,19 +43,19 @@ void MAMConstituentFluxes::set_grids(
// --------------------------------------------------------------------------
// ----------- Atmospheric quantities -------------
// Specific humidity [kg/kg](Require only for building DS)
add_tracer<Required>("qv", grid_, q_unit);
add_tracer<Required>("qv", grid_, q_unit, true);

// Cloud liquid mass mixing ratio [kg/kg](Require only for building DS)
add_tracer<Required>("qc", grid_, q_unit);
add_tracer<Required>("qc", grid_, q_unit, true);

// Cloud ice mass mixing ratio [kg/kg](Require only for building DS)
add_tracer<Required>("qi", grid_, q_unit);
add_tracer<Required>("qi", grid_, q_unit, true);

// Cloud liquid number mixing ratio [1/kg](Require only for building DS)
add_tracer<Required>("nc", grid_, n_unit);
add_tracer<Required>("nc", grid_, n_unit, true);

// Cloud ice number mixing ratio [1/kg](Require only for building DS)
add_tracer<Required>("ni", grid_, n_unit);
add_tracer<Required>("ni", grid_, n_unit, true);

// Temperature[K] at midpoints
add_field<Required>("T_mid", scalar3d_mid, K, grid_name);
Expand Down Expand Up @@ -98,15 +98,16 @@ void MAMConstituentFluxes::set_grids(

// interstitial and cloudborne aerosol tracers of interest: mass (q) and
// number (n) mixing ratios
// NOTE:
// - For interstitial aerosols, we have dynamics advect, but not turbulence.
// - For cloudborne aerosols, DO NOT advect. (for now just a field)
for(int mode = 0; mode < mam_coupling::num_aero_modes(); ++mode) {
// interstitial aerosol tracers of interest: number (n) mixing ratios
const std::string int_nmr_field_name =
mam_coupling::int_aero_nmr_field_name(mode);
add_tracer<Updated>(int_nmr_field_name, grid_, n_unit);
add_tracer<Updated>(int_nmr_field_name, grid_, n_unit, false);

// cloudborne aerosol tracers of interest: number (n) mixing ratios
// NOTE: DO NOT add cld borne aerosols to the "tracer" group as these are
// NOT advected
const std::string cld_nmr_field_name =
mam_coupling::cld_aero_nmr_field_name(mode);
add_field<Updated>(cld_nmr_field_name, scalar3d_mid, n_unit, grid_name);
Expand All @@ -116,11 +117,9 @@ void MAMConstituentFluxes::set_grids(
const std::string int_mmr_field_name =
mam_coupling::int_aero_mmr_field_name(mode, a);
if(not int_mmr_field_name.empty()) {
add_tracer<Updated>(int_mmr_field_name, grid_, q_unit);
add_tracer<Updated>(int_mmr_field_name, grid_, q_unit, false);
}
// (cloudborne) aerosol tracers of interest: mass (q) mixing ratios
// NOTE: DO NOT add cld borne aerosols to the "tracer" group as these are
// NOT advected
const std::string cld_mmr_field_name =
mam_coupling::cld_aero_mmr_field_name(mode, a);
if(not cld_mmr_field_name.empty()) {
Expand All @@ -131,7 +130,7 @@ void MAMConstituentFluxes::set_grids(

for(int g = 0; g < mam_coupling::num_aero_gases(); ++g) {
const std::string gas_mmr_field_name = mam_coupling::gas_mmr_field_name(g);
add_tracer<Updated>(gas_mmr_field_name, grid_, q_unit);
add_tracer<Updated>(gas_mmr_field_name, grid_, q_unit, true);
} // end for loop num gases

} // set_grid
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,19 +71,19 @@ void MAMDryDep::set_grids(

// ----------- Atmospheric quantities -------------
// Specific humidity [kg/kg](Require only for building DS)
add_tracer<Required>("qv", grid_, q_unit);
add_tracer<Required>("qv", grid_, q_unit, true);

// Cloud liquid mass mixing ratio [kg/kg](Require only for building DS)
add_tracer<Required>("qc", grid_, q_unit);
add_tracer<Required>("qc", grid_, q_unit, true);

// Cloud ice mass mixing ratio [kg/kg](Require only for building DS)
add_tracer<Required>("qi", grid_, q_unit);
add_tracer<Required>("qi", grid_, q_unit, true);

// Cloud liquid number mixing ratio [1/kg](Require only for building DS)
add_tracer<Required>("nc", grid_, n_unit);
add_tracer<Required>("nc", grid_, n_unit, true);

// Cloud ice number mixing ratio [1/kg](Require only for building DS)
add_tracer<Required>("ni", grid_, n_unit);
add_tracer<Required>("ni", grid_, n_unit, true);

// Temperature[K] at midpoints
add_field<Required>("T_mid", scalar3d_mid, K, grid_name);
Expand Down Expand Up @@ -153,20 +153,22 @@ void MAMDryDep::set_grids(

// (interstitial) aerosol tracers of interest: mass (q) and number (n) mixing
// ratios
// NOTE: For interstitial aerosols, we have dynamics advect, but not turbulence.
for(int m = 0; m < num_aero_modes; ++m) {
const char *int_nmr_field_name = mam_coupling::int_aero_nmr_field_name(m);

add_tracer<Updated>(int_nmr_field_name, grid_, n_unit);
add_tracer<Updated>(int_nmr_field_name, grid_, n_unit, false);
for(int a = 0; a < mam_coupling::num_aero_species(); ++a) {
const char *int_mmr_field_name =
mam_coupling::int_aero_mmr_field_name(m, a);

if(strlen(int_mmr_field_name) > 0) {
add_tracer<Updated>(int_mmr_field_name, grid_, q_unit);
add_tracer<Updated>(int_mmr_field_name, grid_, q_unit, false);
}
}
}
// (cloud) aerosol tracers of interest: mass (q) and number (n) mixing ratios
// NOTE: For cloudborne aerosols, DO NOT advect.
for(int m = 0; m < num_aero_modes; ++m) {
const char *cld_nmr_field_name = mam_coupling::cld_aero_nmr_field_name(m);

Expand All @@ -184,7 +186,7 @@ void MAMDryDep::set_grids(
// aerosol-related gases: mass mixing ratios
for(int g = 0; g < mam_coupling::num_aero_gases(); ++g) {
const char *gas_mmr_field_name = mam_coupling::gas_mmr_field_name(g);
add_tracer<Updated>(gas_mmr_field_name, grid_, q_unit);
add_tracer<Updated>(gas_mmr_field_name, grid_, q_unit, true);
}

// -------------------------------------------------------------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,19 +81,19 @@ void MAMMicrophysics::set_grids(
// ----------- Atmospheric quantities -------------

// Specific humidity [kg/kg](Require only for building DS)
add_tracer<Required>("qv", grid_, kg/kg); // specific humidity
add_tracer<Required>("qv", grid_, kg/kg, true); // specific humidity

// Cloud liquid mass mixing ratio [kg/kg](Require only for building DS)
add_tracer<Updated>("qc", grid_, kg/kg); // cloud liquid wet mixing ratio
add_tracer<Updated>("qc", grid_, kg/kg, true); // cloud liquid wet mixing ratio

// Cloud ice mass mixing ratio [kg/kg](Require only for building DS)
add_tracer<Required>("qi", grid_, kg/kg); // ice wet mixing ratio
add_tracer<Required>("qi", grid_, kg/kg, true); // ice wet mixing ratio

// Cloud liquid number mixing ratio [1/kg](Require only for building DS)
add_tracer<Updated>("nc", grid_, n_unit); // cloud liquid wet number mixing ratio
add_tracer<Updated>("nc", grid_, n_unit, true); // cloud liquid wet number mixing ratio

// Cloud ice number mixing ratio [1/kg](Require only for building DS)
add_tracer<Required>("ni", grid_, n_unit); // ice number mixing ratio
add_tracer<Required>("ni", grid_, n_unit, true); // ice number mixing ratio

// Temperature[K] at midpoints
add_field<Required>("T_mid", scalar3d_mid, K, grid_name);
Expand Down Expand Up @@ -152,20 +152,22 @@ void MAMMicrophysics::set_grids(

// (interstitial) aerosol tracers of interest: mass (q) and number (n) mixing
// ratios
// NOTE: For interstitial aerosols, we have dynamics advect, but not turbulence.
for(int m = 0; m < nmodes; ++m) {
const char *int_nmr_field_name = mam_coupling::int_aero_nmr_field_name(m);

add_tracer<Updated>(int_nmr_field_name, grid_, n_unit);
add_tracer<Updated>(int_nmr_field_name, grid_, n_unit, false);
for(int a = 0; a < mam_coupling::num_aero_species(); ++a) {
const char *int_mmr_field_name =
mam_coupling::int_aero_mmr_field_name(m, a);

if(strlen(int_mmr_field_name) > 0) {
add_tracer<Updated>(int_mmr_field_name, grid_, kg/kg);
add_tracer<Updated>(int_mmr_field_name, grid_, kg/kg, false);
}
} // for loop species
} // for loop nmodes interstitial
// (cloud) aerosol tracers of interest: mass (q) and number (n) mixing ratios
// NOTE: For cloudborne aerosols, DO NOT advect.
for(int m = 0; m < nmodes; ++m) {
const char *cld_nmr_field_name = mam_coupling::cld_aero_nmr_field_name(m);

Expand All @@ -183,7 +185,7 @@ void MAMMicrophysics::set_grids(
// aerosol-related gases: mass mixing ratios
for(int g = 0; g < mam_coupling::num_aero_gases(); ++g) {
const char *gas_mmr_field_name = mam_coupling::gas_mmr_field_name(g);
add_tracer<Updated>(gas_mmr_field_name, grid_, kg/kg);
add_tracer<Updated>(gas_mmr_field_name, grid_, kg/kg, true);
}

// Creating a Linoz reader and setting Linoz parameters involves reading data
Expand Down
Loading
Loading