CMake error: include could not find detect_openmp

[michalakes]

Description

Compiling ccppV6 as part of NEPTUNE build, we now get an error:

CMake Error at CMakeLists.txt:116 (include)
  include could not find load file:
    detect_openmp

I don't think we had this problem with ccpp v5

Steps to Reproduce

I'm not sure what the non-NEPTUNE to reproduce this is, but it's happening as a result of the command:

    ./build_ccpp.sh sandy /users/michalakes/neptune_atmos "DEBUG=N           \
                    OPENMP=Y HYBRID=N STATIC=Y 32BIT=N" "NO" "NO"

in our NEPTUNE build.

Additional Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

• Machine
• Compiler
• Suite Definition File or Scheme
• Reference other issues or PRs in other repositories that this is related to, and how they are related.

Output

I'm attaching a screenshot of the line 116 in CMakeLists that's referred to in the error message, above.

[dustinswales]

@michalakes The snippet of code above from CMakeLists.txt appears to be from the v5 release? CMakeLists has been updated for CCPP v6.

[michalakes]

@michalakes The snippet of code above from CMakeLists.txt appears to be from the v5 release? CMakeLists has been updated for CCPP v6.

Thanks, Dustin. I think with NEPTUNE we've got a V6/V5 chimera at the CMake level. I found a workaround, for now, that involved recovering the errant detect_openmp.cmake from V5 and adding it back to our patchwork around V6. I think it's our problem so this issue can be closed, but before I do that, a follow up, please:

Background: We need for CCPP to be compiled without OpenMP, even when NEPTUNE is compiled with OpenMP. For the Intel compiler, we do need the -auto flag but not -qopenmp. That's because we have the threading at a higher level, above the calls to CCPP. We expect the code inside CCPP to run single threaded on each higher-level thread that it's called. The problem we ran into in V6 was that CMake was seeing OPENMP set to true in the environment and then compiling it with -qopenmp. After a couple days of hunting I see that we had put a line into the V5 detect_openmp.cmake that specifically omitted -qopenmp from the Fortran flags. When detect_openmp.cmake went missing, so did that work-around.

So the questions:

1. Does the new V6 CMake apparatus have a way of saying not to compile with OpenMP flags but with -auto?
2. Does CCPP physics actually build under the Intel when -qopenmp is specified? I see cascades of error messages having to do with !$omp target directives.

If you prefer, I could close the issue now and we could discuss this offline. Thanks for your reply!

John

[michalakes]

I am closing this issue. We found a fix that does not require the detect_openmp.cmake file or any other modifications to submodules that NEPTUNE gets from CCPP.

[dustinswales]

@michalakes Sorry for not commenting sooner.
Currently the cmake allows for schemes to be compiler w/o openmp, but this is done for each file and they need to be listed in CMakeLists.txt. For example, all of the rte-rrtmgp physics is compiled w/o openmp. I suppose this is not what you want for NEPTUNE?, but rather to switch off the openmp for all the schemes. If so, here's a proposed solution that relies on an environmental variable, DISABLE_CCPP_OPENMP, that turns off the openmp in the ccpp-physics build, but not in the overlaying host (tested this in the SCM). There are other ways to do this, but this seemed the least invasive.
Also, '-auto' flag is set for Intel in the SCM. Not sure about fv3, but I will take a look.