Set amrex.omp_threads = "nosmt"

Improve the performance on Intel and AMD CPUs by default:
Avoid oversubscribing the physical number of CPU threads, as is
default in OpenMP.

Author: ax3l

docs/source/usage/parameters.rst

@@ -33,6 +33,11 @@ Overall simulation parameters
     For all regular ImpactX operations, we therefore do explicit memory transfers without the need for managed memory and thus changed the AMReX default to false.
     `Please also see the documentation in AMReX <https://amrex-codes.github.io/amrex/docs_html/GPU.html#inputs-parameters>`__.
 
+* ``amrex.omp_threads``  (``system``, ``nosmt`` or positive integer; default is ``nosmt``)
+    An integer number can be set in lieu of the ``OMP_NUM_THREADS`` environment variable to control the number of OpenMP threads to use for the ``OMP`` compute backend on CPUs.
+    By default, we use the ``nosmt`` option, which overwrites the OpenMP default of spawning one thread per logical CPU core, and instead only spawns a number of threads equal to the number of physical CPU cores on the machine.
+    If set, the environment variable ``OMP_NUM_THREADS`` takes precedence over ``system`` and ``nosmt``, but not over integer numbers set in this option.
+
 * ``amrex.abort_on_unused_inputs`` (``0`` or ``1``; default is ``0`` for false)
     When set to ``1``, this option causes the simulation to fail *after* its completion if there were unused parameters.
     It is mainly intended for continuous integration and automated testing to check that all tests and inputs are adapted to API changes.

src/initialization/InitParser.cpp

@@ -25,6 +25,10 @@ namespace impactx::initialization
         bool the_arena_is_managed = false; // AMReX' default: true
         pp_amrex.queryAdd("the_arena_is_managed", the_arena_is_managed);
 
+        // https://amrex-codes.github.io/amrex/docs_html/InputsComputeBackends.html
+        std::string omp_threads = "nosmt"; // AMReX' default: system
+        pp_amrex.queryAdd("omp_threads", omp_threads);
+
         // Here we override the default tiling option for particles, which is always
         // "false" in AMReX, to "false" if compiling for GPU execution and "true"
         // if compiling for CPU.