From 883ebcb2509df68702a7207bdd410926f20e3e9c Mon Sep 17 00:00:00 2001 From: Dom Heinzeller Date: Mon, 6 Nov 2023 20:06:34 -0700 Subject: [PATCH] release/1.5.1: add fms@2023.03, site config updates round 3 (#855) * Update site configs for NOAA Parallel Works * Update casper site config and build documentation * Update container specs in configs/containers/README.md * Update Hercules site config (gnu@12, mvapich2@2.3.7) * Update doc/source/PreConfiguredSites.rst for casper, hera, gaea-c4, jet * Add fms@2023.03 in spack submodule, add to unified-dev and skylab-dev templates * Update configs/containers after tagging spack submodule as spack-stack-1.5.1 * Update documentation for AWS AMI release 1.5.1 * Update macOS section in doc/source/NewSiteConfigs.rst to match Linux section --- .gitmodules | 2 +- configs/common/packages.yaml | 1 + configs/containers/README.md | 13 +- .../containers/docker-ubuntu-clang-mpich.yaml | 2 +- .../containers/docker-ubuntu-gcc-openmpi.yaml | 2 +- .../containers/docker-ubuntu-intel-impi.yaml | 2 +- configs/sites/casper/compilers.yaml | 25 ++-- configs/sites/casper/packages.yaml | 113 +++++++++--------- configs/sites/hercules/compilers.yaml | 31 ++--- configs/sites/hercules/packages.yaml | 13 +- configs/sites/noaa-aws/packages.yaml | 5 + configs/sites/noaa-azure/packages.yaml | 13 +- configs/sites/noaa-gcloud/packages.yaml | 5 + configs/templates/skylab-dev/spack.yaml | 3 +- configs/templates/unified-dev/spack.yaml | 1 + doc/source/MaintainersSection.rst | 66 ++-------- doc/source/NewSiteConfigs.rst | 34 +++--- doc/source/PreConfiguredSites.rst | 84 +++++++------ spack | 2 +- 19 files changed, 175 insertions(+), 242 deletions(-) diff --git a/.gitmodules b/.gitmodules index c616b64db..469902dc4 100644 --- a/.gitmodules +++ b/.gitmodules @@ -3,7 +3,7 @@ #url = https://github.com/spack/spack #branch = develop url = https://github.com/jcsda/spack - branch = release/1.5.1 + branch = spack-stack-1.5.1 [submodule "doc/CMakeModules"] path = doc/CMakeModules url = https://github.com/noaa-emc/cmakemodules diff --git a/configs/common/packages.yaml b/configs/common/packages.yaml index 4644ec2f6..dcfe6ccac 100644 --- a/configs/common/packages.yaml +++ b/configs/common/packages.yaml @@ -28,6 +28,7 @@ variants: +pic # Newer versions of CDO require the C++-17 standard, which doesn't # work with all compilers that are currently in use in spack-stack + # When making changes here, also check Hercules/Casper site configs cdo: version: ['2.0.5'] variants: ~openmp diff --git a/configs/containers/README.md b/configs/containers/README.md index 857528991..adba02191 100644 --- a/configs/containers/README.md +++ b/configs/containers/README.md @@ -2,25 +2,24 @@ To avoid hardcoding specs in the generic container recipes, we keep the specs list empty (`specs: []`) and manually add the specs for the particular spack-stack release and application as listed below, *after* running `spack stack create ctr`. -### spack-stack-1.5.0 / skylab-6.0.0 containers for fv3-jedi and mpas-jedi (but not for ufs-jedi) +### spack-stack-1.5.1 / skylab-6.1.0 containers for fv3-jedi and mpas-jedi (but not for ufs-jedi) ``` specs: [base-env@1.0.0, jedi-base-env@1.0.0, ewok-env@1.0.0, jedi-fv3-env@1.0.0, jedi-mpas-env@1.0.0, bacio@2.4.1, bison@3.8.2, bufr@12.0.0, ecbuild@3.7.2, eccodes@2.27.0, ecflow@5, eckit@1.24.4, ecmwf-atlas@0.35.0 +fckit +ectrans +tesselation +fftw, fiat@1.2.0, ectrans@1.2.0 +fftw, - eigen@3.4.0, fckit@0.11.0, fms@release-jcsda, g2@3.4.5, g2tmpl@1.10.0, gftl-shared@1.5.0, + eigen@3.4.0, fckit@0.11.0, fms@release-jcsda, g2@3.4.5, g2tmpl@1.10.0, gftl-shared@1.6.1, gsibec@1.1.3, hdf@4.2.15, hdf5@1.14.0, ip@4.3.0, jasper@2.0.32, jedi-cmake@1.4.0, libpng@1.6.37, nccmp@1.9.0.1, netcdf-c@4.9.2, netcdf-cxx4@4.3.1, netcdf-fortran@4.6.0, nlohmann-json@3.10.5, nlohmann-json-schema-validator@2.1.0, parallelio@2.5.10, parallel-netcdf@1.12.2, py-eccodes@1.4.2, py-f90nml@1.4.3, py-gitpython@3.1.27, py-h5py@3.7.0, py-numpy@1.22.3, py-pandas@1.5.3, py-pip, py-pyyaml@5.4.1, py-scipy@1.9.3, py-shapely@1.8.0, py-xarray@2022.3.0, - sp@2.3.3, udunits@2.2.28, w3nco@2.4.1, w3emc@2.10.0, nco@5.0.6, esmf@8.4.2, mapl@2.35.2, - yafyaml@0.5.1, zlib@1.2.13, zstd@1.5.2, odc@1.4.6, shumlib@macos_clang_linux_intel_port, + sp@2.3.3, udunits@2.2.28, w3nco@2.4.1, w3emc@2.10.0, nco@5.0.6, esmf@8.5.0, mapl@2.40.3, + zlib@1.2.13, zstd@1.5.2, odc@1.4.6, shumlib@macos_clang_linux_intel_port, awscli@1.27.84, py-globus-cli@3.16.0] # Notes: - # 1. Remove mapl@2.35.2 from clang/mpich container, because mapl doesn't work with mpich@4 - # 2. Don't build CRTM by default so that it gets built in the JEDI bundles - # 3. Comment out for now until build problems are solved + # 1. Don't build CRTM by default so that it gets built in the JEDI bundles + # 2. Comment out for now until build problems are solved # https://github.com/jcsda/spack-stack/issues/522 # py-mysql-connector-python@8.0.32 ``` diff --git a/configs/containers/docker-ubuntu-clang-mpich.yaml b/configs/containers/docker-ubuntu-clang-mpich.yaml index c12b12d0e..a63f52f28 100644 --- a/configs/containers/docker-ubuntu-clang-mpich.yaml +++ b/configs/containers/docker-ubuntu-clang-mpich.yaml @@ -110,7 +110,7 @@ spack: os: ubuntu:20.04 spack: url: https://github.com/jcsda/spack - ref: release/1.5.1 + ref: spack-stack-1.5.1 resolve_sha: false # Whether or not to strip binaries diff --git a/configs/containers/docker-ubuntu-gcc-openmpi.yaml b/configs/containers/docker-ubuntu-gcc-openmpi.yaml index cc72f9d35..3aa0b8622 100644 --- a/configs/containers/docker-ubuntu-gcc-openmpi.yaml +++ b/configs/containers/docker-ubuntu-gcc-openmpi.yaml @@ -98,7 +98,7 @@ spack: os: ubuntu:20.04 spack: url: https://github.com/jcsda/spack - ref: release/1.5.1 + ref: spack-stack-1.5.1 resolve_sha: false # Whether or not to strip binaries diff --git a/configs/containers/docker-ubuntu-intel-impi.yaml b/configs/containers/docker-ubuntu-intel-impi.yaml index c81a79838..1b01bb603 100644 --- a/configs/containers/docker-ubuntu-intel-impi.yaml +++ b/configs/containers/docker-ubuntu-intel-impi.yaml @@ -116,7 +116,7 @@ spack: os: ubuntu:20.04 spack: url: https://github.com/jcsda/spack - ref: release/1.5.1 + ref: spack-stack-1.5.1 resolve_sha: false # Whether or not to strip binaries diff --git a/configs/sites/casper/compilers.yaml b/configs/sites/casper/compilers.yaml index c83881757..d223f6bb5 100644 --- a/configs/sites/casper/compilers.yaml +++ b/configs/sites/casper/compilers.yaml @@ -1,24 +1,15 @@ compilers:: - compiler: - spec: intel@2022.0.2 + spec: gcc@12.2.0 paths: - cc: /glade/u/apps/opt/intel/2022.1/compiler/2022.0.2/linux/bin/intel64/icc - cxx: /glade/u/apps/opt/intel/2022.1/compiler/2022.0.2/linux/bin/intel64/icpc - f77: /glade/u/apps/opt/intel/2022.1/compiler/2022.0.2/linux/bin/intel64/ifort - fc: /glade/u/apps/opt/intel/2022.1/compiler/2022.0.2/linux/bin/intel64/ifort + cc: /glade/u/apps/casper/23.10/spack/opt/spack/gcc/12.2.0/pucl/bin/gcc + cxx: /glade/u/apps/casper/23.10/spack/opt/spack/gcc/12.2.0/pucl/bin/g++ + f77: /glade/u/apps/casper/23.10/spack/opt/spack/gcc/12.2.0/pucl/bin/gfortran + fc: /glade/u/apps/casper/23.10/spack/opt/spack/gcc/12.2.0/pucl/bin/gfortran flags: {} - operating_system: centos7 + operating_system: opensuse15 target: x86_64 modules: - - intel/2022.1 - environment: - prepend_path: - PATH: '/glade/work/epicufsrt/contrib/spack-stack/casper/gcc-10.1.0/bin' - CPATH: '/glade/work/epicufsrt/contrib/spack-stack/casper/gcc-10.1.0/include' - LD_LIBRARY_PATH: '/glade/u/apps/opt/intel/2022.1/compiler/latest/linux/compiler/lib/intel64_lin:/glade/u/apps/opt/intel/2022.1/mpi/2021.5.1/libfabric/lib:/glade/work/epicufsrt/contrib/spack-stack/casper/gcc-10.1.0/lib64' - set: - INTEL_LICENSE_FILE: '28518@128.117.177.41' - LM_LICENSE_FILE: '28518@128.117.177.41' - I_MPI_ROOT: '/glade/u/apps/opt/intel/2022.1/mpi/2021.5.1' - + - gcc/12.2.0 + environment: {} extra_rpaths: [] diff --git a/configs/sites/casper/packages.yaml b/configs/sites/casper/packages.yaml index fef8acbb5..57b28a16b 100644 --- a/configs/sites/casper/packages.yaml +++ b/configs/sites/casper/packages.yaml @@ -1,32 +1,36 @@ packages: all: - compiler:: [intel@2022.0.2] + compiler:: [gcc@12.2.0] providers: - mpi:: [intel-oneapi-mpi@2021.5.1] + mpi:: [openmpi@4.1.6] ### MPI, Python, MKL mpi: buildable: False - intel-oneapi-mpi: + openmpi: externals: - - spec: intel-oneapi-mpi@2021.5.1%intel@2022.0.2 - prefix: /glade/u/apps/opt/intel/2022.1 + - spec: openmpi@4.1.6%gcc@12.2.0+cuda~cxx~cxx_exceptions~java+lustre~memchecker+pmi+static~wrapper-rpath fabrics=ucx schedulers=tm + prefix: /glade/u/apps/casper/23.10/spack/opt/spack/openmpi/4.1.6/gcc/12.2.0/yia4 modules: - - intel-oneapi-mpi/2021.5.1.lua + - openmpi/4.1.6 + - ucx/1.14.1 + - cuda/12.2.1 -### Modification of common packages - # Default gdal (3.7.0) doesn't build on Casper with intel@2022.0.2 - gdal: - version:: ['3.6.3'] +### Modifications of common packages + # Version 2.0.5 doesn't compile on Hercules with gcc + cdo + version:: ['2.2.0'] ### All other external packages listed alphabetically autoconf: externals: - - spec: autoconf@2.69 - prefix: /usr + - spec: autoconf@2.71 + prefix: /glade/u/apps/casper/23.10/opt/view automake: externals: - - spec: automake@1.13.4 + - spec: automake@1.16.5 + prefix: /glade/u/apps/casper/23.10/opt/view + - spec: automake@1.15.1 prefix: /usr berkeley-db: externals: @@ -34,12 +38,16 @@ packages: prefix: /usr binutils: externals: - - spec: binutils@2.27.43 + - spec: binutils@2.41.0 prefix: /usr bzip2: externals: - spec: bzip2@1.0.6 prefix: /usr + coreutils: + externals: + - spec: coreutils@8.32 + prefix: /usr cmake: externals: - spec: cmake@3.16.8 @@ -53,11 +61,13 @@ packages: prefix: /usr cvs: externals: - - spec: cvs@1.11.23 + - spec: cvs@1.12.13 prefix: /usr curl: externals: - - spec: curl@7.29.0+gssapi+ldap+nghttp2 + - spec: curl@8.1.2+nghttp2 + prefix: /glade/u/apps/casper/23.10/opt/view + - spec: curl@8.0.1+gssapi+ldap+nghttp2 prefix: /usr diffutils: externals: @@ -65,7 +75,7 @@ packages: prefix: /usr doxygen: externals: - - spec: doxygen@1.8.5+graphviz~mscgen + - spec: diffutils@3.6 prefix: /usr ecflow: buildable: False @@ -74,23 +84,19 @@ packages: prefix: /glade/work/epicufsrt/contrib/spack-stack/casper/ecflow-5.8.4 modules: - ecflow/5.8.4 - file: - externals: - - spec: file@5.11 - prefix: /usr findutils: externals: - - spec: findutils@4.5.11 + - spec: findutils@4.8.0 prefix: /usr flex: # Must set buildable: false to avoid duplicate packages - buildable: false + #buildable: false externals: - - spec: flex@2.5.37+lex - prefix: /usr + - spec: flex@2.6.4+lex + prefix: /glade/u/apps/casper/23.10/opt/view gawk: externals: - - spec: gawk@4.0.2 + - spec: gawk@4.2.1 prefix: /usr gettext: externals: @@ -102,21 +108,21 @@ packages: prefix: /usr git: externals: - - spec: git@2.33.1 - modules: - - git/2.33.1 + - spec: git@2.41.0~tcltk + prefix: /glade/u/apps/casper/23.10/opt/view + - spec: git@2.35.3+tcltk + prefix: /usr git-lfs: externals: - - spec: git-lfs@3.0.2 - modules: - - git/2.33.1 + - spec: git-lfs@3.3.0 + prefix: /glade/u/apps/casper/23.10/opt/view gmake: externals: - - spec: gmake@3.82 + - spec: gmake@4.2.1 prefix: /usr groff: externals: - - spec: groff@1.22.2 + - spec: groff@1.22.4 prefix: /usr krb5: buildable: False @@ -131,15 +137,13 @@ packages: variants: ~gssapi libtool: externals: - - spec: libtool@2.4.2 - prefix: /usr - libxpm: - externals: - - spec: libxpm@4.11.0 + - spec: libtool@2.4.7 + prefix: /glade/u/apps/casper/23.10/opt/view + - spec: libtool@2.4.6 prefix: /usr m4: externals: - - spec: m4@1.4.16 + - spec: m4@1.4.18 prefix: /usr mysql: buildable: False @@ -150,34 +154,31 @@ packages: - mysql/8.0.31 ncurses: externals: - - spec: ncurses@5.9.20130511+termlib abi=5 - prefix: /usr + - spec: ncurses@6.4.20221231+termlib abi=6 + prefix: /glade/u/apps/casper/23.10/opt/view # Old re2c on Casper unable to build newer versions of ninja ninja: version:: ['1.10.2'] openssh: externals: - - spec: openssh@7.4p1 + - spec: openssh@8.4p1 prefix: /usr openssl: externals: - - spec: openssl@1.0.2k-fips + - spec: openssl@1.1.1l prefix: /usr - # Pin patchelf to 0.15.0 for Intel compiler (no C++-17 features) - #patchelf: - # version:: ['0.15.0'] perl: externals: - spec: perl@5.16.3~cpanm+shared+threads prefix: /usr pkg-config: externals: - - spec: pkg-config@0.27.1 + - spec: pkg-config@0.29.2 prefix: /usr qt: externals: - - spec: qt@5.15.2 - prefix: /glade/work/epicufsrt/contrib/spack-stack/casper/qt-5.15.2 + - spec: qt@5.14.2 + prefix: /glade/u/apps/casper/23.10/opt/view rsync: externals: - spec: rsync@3.1.2 @@ -192,19 +193,17 @@ packages: prefix: /usr subversion: externals: - - spec: subversion@1.7.14 - prefix: /usr - swig: - externals: - - spec: swig@2.0.10 + - spec: subversion@1.14.1 prefix: /usr tar: externals: - - spec: tar@1.26 + - spec: tar@1.34 prefix: /usr texinfo: externals: - - spec: texinfo@5.1 + - spec: texinfo@7.0.3 + prefix: /glade/u/apps/casper/23.10/opt/view + - spec: texinfo@6.5 prefix: /usr texlive: externals: diff --git a/configs/sites/hercules/compilers.yaml b/configs/sites/hercules/compilers.yaml index 4b1f3fda5..a86be4858 100644 --- a/configs/sites/hercules/compilers.yaml +++ b/configs/sites/hercules/compilers.yaml @@ -13,34 +13,17 @@ compilers: - intel-oneapi-compilers/2023.1.0 environment: {} extra_rpaths: [] -# Spack gets confused if both intel and oneapi are present -#- compiler: -# spec: oneapi@2022.2.1 -# paths: -# cc: /apps/spack-managed/gcc-11.3.1/intel-oneapi-compilers-2022.2.1-z2sjni66fcyqcsamnoccgb7c77mn37qj/compiler/2022.2.1/linux/bin/icx -# cxx: /apps/spack-managed/gcc-11.3.1/intel-oneapi-compilers-2022.2.1-z2sjni66fcyqcsamnoccgb7c77mn37qj/compiler/2022.2.1/linux/bin/icpx -# # For now ... -# f77: /apps/spack-managed/gcc-11.3.1/intel-oneapi-compilers-2022.2.1-z2sjni66fcyqcsamnoccgb7c77mn37qj/compiler/2022.2.1/linux/bin/intel64/ifort -# fc: /apps/spack-managed/gcc-11.3.1/intel-oneapi-compilers-2022.2.1-z2sjni66fcyqcsamnoccgb7c77mn37qj/compiler/2022.2.1/linux/bin/intel64/ifort -# #f77: /apps/spack-managed/gcc-11.3.1/intel-oneapi-compilers-2022.2.1-z2sjni66fcyqcsamnoccgb7c77mn37qj/compiler/2022.2.1/linux/bin/ifx -# #fc: /apps/spack-managed/gcc-11.3.1/intel-oneapi-compilers-2022.2.1-z2sjni66fcyqcsamnoccgb7c77mn37qj/compiler/2022.2.1/linux/bin/ifx -# flags: {} -# operating_system: rocky9 -# target: x86_64 -# modules: -# - intel-oneapi-compilers/2022.2.1 -# environment: {} -# extra_rpaths: [] - compiler: - spec: gcc@11.3.1 + spec: gcc@12.2.0 paths: - cc: /usr/bin/gcc - cxx: /usr/bin/g++ - f77: /usr/bin/gfortran - fc: /usr/bin/gfortran + cc: /apps/spack-managed/gcc-11.3.1/gcc-12.2.0-7cu3qahzhsxpauy4jlnsbcqmlbkxbbbo/bin/gcc + cxx: /apps/spack-managed/gcc-11.3.1/gcc-12.2.0-7cu3qahzhsxpauy4jlnsbcqmlbkxbbbo/bin/g++ + f77: /apps/spack-managed/gcc-11.3.1/gcc-12.2.0-7cu3qahzhsxpauy4jlnsbcqmlbkxbbbo/bin/gfortran + fc: /apps/spack-managed/gcc-11.3.1/gcc-12.2.0-7cu3qahzhsxpauy4jlnsbcqmlbkxbbbo/bin/gfortran flags: {} operating_system: rocky9 target: x86_64 - modules: [] + modules: + - gcc/12.2.0 environment: {} extra_rpaths: [] diff --git a/configs/sites/hercules/packages.yaml b/configs/sites/hercules/packages.yaml index 33eace390..ba90ee657 100644 --- a/configs/sites/hercules/packages.yaml +++ b/configs/sites/hercules/packages.yaml @@ -1,6 +1,6 @@ packages: all: - compiler:: [intel@2021.9.0, gcc@11.3.1] + compiler:: [intel@2021.9.0, gcc@12.2.0] providers: mpi:: [intel-oneapi-mpi@2021.9.0, mvapich2@2.3.7] @@ -15,13 +15,16 @@ packages: - intel-oneapi-mpi/2021.9.0 mvapich2: externals: - - spec: mvapich2@2.3.7%gcc@11.3.1~cuda~debug~regcache~wrapperrpath process_managers=slurm - prefix: /work/noaa/epic/role-epic/spack-stack/hercules/mvapich2-2.3.7/gcc-11.3.1 + - spec: mvapich2@2.3.7%gcc@12.2.0~cuda~debug~regcache~wrapperrpath process_managers=slurm + prefix: /work/noaa/epic/role-epic/spack-stack/hercules/mvapich2-2.3.7/gcc-12.2.0 modules: - - slurm/22.05.8 - - mvapich2/2.3.7 + - gcc/12.2.0 + - mvapich2/2.3.7-gcc12 ### Modifications of common packages + # Version 2.0.5 doesn't compile on Hercules with gcc + cdo: + version:: ['2.2.0'] # Version 2.0.8 doesn't compile on Hercules wgrib2: version:: ['3.1.1'] diff --git a/configs/sites/noaa-aws/packages.yaml b/configs/sites/noaa-aws/packages.yaml index fffec16e4..bb348d5d2 100644 --- a/configs/sites/noaa-aws/packages.yaml +++ b/configs/sites/noaa-aws/packages.yaml @@ -28,6 +28,11 @@ packages: modules: - openmpi/3.1.4 +### Modifications of common packages + # Pin flex to avoid duplicate packages + flex: + version: ['2.6.4'] + ### All other external packages listed alphabetically bash: externals: diff --git a/configs/sites/noaa-azure/packages.yaml b/configs/sites/noaa-azure/packages.yaml index 781d2712d..a2c78fd7f 100644 --- a/configs/sites/noaa-azure/packages.yaml +++ b/configs/sites/noaa-azure/packages.yaml @@ -28,6 +28,11 @@ packages: modules: - openmpi/3.1.4 +### Modifications of common packages + # Pin flex to avoid duplicate packages + flex: + version: ['2.6.4'] + ### All other external packages listed alphabetically autoconf: externals: @@ -45,10 +50,6 @@ packages: externals: - spec: berkeley-db@5.3.21 prefix: /usr - bison: - externals: - - spec: bison@3.0.4 - prefix: /usr cmake: externals: - spec: cmake@3.27.2 @@ -78,10 +79,6 @@ packages: externals: - spec: findutils@4.5.11 prefix: /usr - flex: - externals: - - spec: flex@2.5.37+lex - prefix: /usr gawk: externals: - spec: gawk@4.0.2 diff --git a/configs/sites/noaa-gcloud/packages.yaml b/configs/sites/noaa-gcloud/packages.yaml index 28c3e0831..c5a7514ae 100644 --- a/configs/sites/noaa-gcloud/packages.yaml +++ b/configs/sites/noaa-gcloud/packages.yaml @@ -28,6 +28,11 @@ packages: modules: - openmpi/3.1.4 +### Modifications of common packages + # Pin flex to avoid duplicate packages + flex: + version: ['2.6.4'] + ### All other external packages listed alphabetically bash: externals: diff --git a/configs/templates/skylab-dev/spack.yaml b/configs/templates/skylab-dev/spack.yaml index 6d1011589..49a9e93e5 100644 --- a/configs/templates/skylab-dev/spack.yaml +++ b/configs/templates/skylab-dev/spack.yaml @@ -19,7 +19,8 @@ spack: # Various fms tags (list all to avoid duplicate packages) - fms@release-jcsda - fms@2023.01 - - fms@2023.02 + - fms@2023.02.01 + - fms@2023.03 # Various crtm tags (list all to avoid duplicate packages) - crtm@2.4.0 diff --git a/configs/templates/unified-dev/spack.yaml b/configs/templates/unified-dev/spack.yaml index af2abd5f6..6c303be44 100644 --- a/configs/templates/unified-dev/spack.yaml +++ b/configs/templates/unified-dev/spack.yaml @@ -29,6 +29,7 @@ spack: - fms@release-jcsda - fms@2023.01 - fms@2023.02.01 + - fms@2023.03 # Various crtm tags (list all to avoid duplicate packages) - crtm@2.4.0 diff --git a/doc/source/MaintainersSection.rst b/doc/source/MaintainersSection.rst index 6f17482cf..d6d58efff 100644 --- a/doc/source/MaintainersSection.rst +++ b/doc/source/MaintainersSection.rst @@ -137,7 +137,7 @@ Create modulefile ``/lustre/f2/pdata/esrl/gsd/spack-stack/modulefiles/ecflow/5.8 cmake .. -DPython3_EXECUTABLE=`which python3` -DCMAKE_INSTALL_PREFIX=/path/to/ecflow/installation 2>&1 | tee log.cmake .. note:: - Finally, on Derecho (or any other system with ``gcc@12.2.0``), one needs to patch file ``ecflow-5.8.4/src/ecFlow-5.8.4-Source/ACore/src/Passwd.cpp`` by adding ``#include `` below line ``#include "Passwd.hpp"`` before running ``make``. + Finally, on Casper, Derecho, or any other system with ``gcc@12.2.0``, one needs to patch file ``ecflow-5.8.4/src/ecFlow-5.8.4-Source/ACore/src/Passwd.cpp`` by adding ``#include `` below line ``#include "Passwd.hpp"`` before running ``make``. .. _MaintainersSection_MySQL: @@ -258,20 +258,21 @@ mysql ``mysql`` must be installed separately from ``spack`` using a binary tarball provided by the MySQL community. Follow the instructions in :numref:`Section %s ` to install ``mysql`` in ``/work/noaa/epic-ps/role-epic-ps/spack-stack/mysql-8.0.31-hercules``. mvapich2 - Because of difficulties with ``openmpi`` on Hercules, we build ``mvapich2``. It is necessary to either load ``qt`` to use a consistent ``zlib``, or to load ``zlib`` directly (check the ``qt`` module). Create modulefile ``mvapich2`` from template ``doc/modulefile_templates/mvapich2``. + Because of difficulties with ``openmpi`` on Hercules, we build ``mvapich2``. It is necessary to either load ``qt`` to use a consistent ``zlib``, or to load ``zlib`` directly (check the ``qt`` module). Create modulefile ``mvapich2`` from template ``doc/modulefile_templates/mvapich2``. **Important:** We identified a bug in ``gcc@11`` + ``mvapich2@2.3.7`` in MPI allgather operations. It is therefore necessary to switch to a newer GCC compiler. .. code-block:: console module purge + module load gcc/12.2.0 module load zlib/1.2.13 module load ucx/1.13.1 - module load slurm/22.05.8 + module load slurm/23.02.6 FFLAGS=-fallow-argument-mismatch ./configure \ --prefix=/work/noaa/epic/role-epic/spack-stack/hercules/mvapich-2.3.7/gcc-11.3.1 \ --with-pmi=pmi2 \ --with-pm=slurm \ - --with-slurm-include=/opt/slurm-22.05.8/include \ - --with-slurm-lib=/opt/slurm-22.05.8/lib \ + --with-slurm-include=/opt/slurm-23.02.6/include \ + --with-slurm-lib=/opt/slurm-23.02.6/lib \ 2>&1 | tee log.config./configure make VERBOSE=1 -j4 make check @@ -414,65 +415,14 @@ NCAR-Wyoming Casper On Casper, there are problems with newer versions of the Intel compiler/MPI library when trying to run MPI jobs with just one task (``mpiexec -np 1``) - for JEDI, job hangs forever in a particular MPI communication call in oops. This is why an older version Intel 19 is used here. -gcc - CISL does not provide a newer GNU compiler for Casper, and the Cheyenne compiler has problems with missing symbols. Build ``gcc`` etc. as follows and create a module file from template ``gcc`` in ``/glade/work/epicufsrt/contrib/spack-stack/casper/modulefiles``. - -.. code-block:: console - - module purge - mkdir -p /glade/work/epicufsrt/contrib/spack-stack/casper/gcc-10.1.0/src - cd /glade/work/epicufsrt/contrib/spack-stack/casper/gcc-10.1.0/src - wget http://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-10.1.0/gcc-10.1.0.tar.gz - tar -xvzf gcc-10.1.0.tar.gz - cd gcc-10.1.0/ - ./contrib/download_prerequisites - ./configure \ - --prefix=/glade/work/epicufsrt/contrib/spack-stack/casper/gcc-10.1.0 \ - --disable-multilib --enable-languages=c,c++ 2>&1 | tee log.config - make -j8 2>&1 | tee log.make - make install 2>&1 | tee log.install - # create modulefile - -intel-oneapi-mpi (module only) - The ``impi/2022.1`` module provided by CISL has the wrong ``I_MPI_ROOT`` and ``MPI_ROOT`` values. Copy the file to ``/glade/work/epicufsrt/contrib/spack-stack/casper/modulefiles/intel-oneapi-mpi/2021.5.1.lua`` and remove the offending entries, as well as the unnecessary ``ncarcompilers`` logic at the end. - -qt (qt@5) - The default ``qt@5`` in ``/usr`` is incomplete and thus insufficient for building ``ecflow``. Follow these instructions to build ``qt@5.15.2`` using ``gcc@10.10.0``. See also https://wiki.qt.io/Building_Qt_5_from_Git#Getting_the_source_code for building qt from source. - -.. code-block:: console - - module purge - export LMOD_TMOD_FIND_FIRST=yes - module use /glade/work/epicufsrt/contrib/spack-stack/casper/modulefiles - module load gcc/10.1.0 - mkdir -p /glade/work/epicufsrt/contrib/spack-stack/casper/qt-5.15.2/src - cd /glade/work/epicufsrt/contrib/spack-stack/casper/qt-5.15.2/src - git clone https://code.qt.io/qt/qt5.git - cd qt5/ - git checkout 5.12 - perl init-repository 2>&1 | tee log.init-repository - cd .. - mkdir qt5-build - cd qt5-build - ../qt5/configure -opensource -nomake examples -nomake tests \ - -prefix "/glade/work/epicufsrt/contrib/spack-stack/casper/qt-5.15.2" 2>&1 | tee log.config - make -j4 2>&1 | tee log.make - make install 2>&1 | tee log.install - # If errors occur during the installation of qtlocation, ignore. This is one of the last steps - # and not needed for ecflow (consider not building this module in the first place ... todo). - ecflow - ``ecFlow`` must be built manually using the GNU compilers and linked against a static ``boost`` library. After installing `qt5`, and loading the following modules, follow the instructions in :numref:`Section %s `. + ``ecFlow`` must be built manually using the GNU compilers and linked against a static ``boost`` library. After loading the following modules, follow the instructions in :numref:`Section %s `. .. code-block:: console module purge export LMOD_TMOD_FIND_FIRST=yes - module use /glade/work/epicufsrt/contrib/spack-stack/casper/modulefiles - module load gnu/10.1.0 - module load python/3.7.9 - module load qt/5.15.2 - module load cmake/3.18.2 + module load gnu/12.2.0 mysql ``mysql`` must be installed separately from ``spack`` using a binary tarball provided by the MySQL community. Follow the instructions in :numref:`Section %s ` to install ``mysql`` in ``/glade/work/epicufsrt/contrib/spack-stack/casper/mysql-8.0.31``. diff --git a/doc/source/NewSiteConfigs.rst b/doc/source/NewSiteConfigs.rst index babc87b30..0e9d011f5 100644 --- a/doc/source/NewSiteConfigs.rst +++ b/doc/source/NewSiteConfigs.rst @@ -269,9 +269,9 @@ Remember to activate the ``lua`` module environment and have MacTeX in your sear .. code-block:: console - vi envs/unified-env.mymacos/spack.yaml - vi envs/unified-env.mymacos/common/*.yaml - vi envs/unified-env.mymacos/site/*.yaml + vi spack.yaml + vi common/*.yaml + vi site/*.yaml 10. Process the specs and install @@ -283,8 +283,8 @@ See the :ref:`documentation ` for usage information including .. code-block:: console spack concretize 2>&1 | tee log.concretize - util/show_duplicate_packages.py -d [-c] log.concretize - spack install [--verbose] [--fail-fast] + ${SPACK_STACK_DIR}/util/show_duplicate_packages.py -d [-c] log.concretize + spack install [--verbose] [--fail-fast] 2>&1 | tee log.install 11. Create lmod module files @@ -301,7 +301,7 @@ See the :ref:`documentation ` for usage information including .. note:: Unlike preconfigured environments and linux environments, MacOS users typically need to activate lmod's ``module`` tool within each shell session. This can be done by running ``source $HOMEBREW_ROOT/opt/lmod/init/profile`` -13. You now have a spack-stack environment that can be accessed by running ``module use ./envs/unified-env.mymacos/install/modulefiles/Core``. The modules defined here can be loaded to build and run code as described in :numref:`Section %s `. +13. You now have a spack-stack environment that can be accessed by running ``module use ${SPACK_STACK_DIR}/envs/unified-env.mymacos/install/modulefiles/Core``. The modules defined here can be loaded to build and run code as described in :numref:`Section %s `. .. _NewSiteConfigs_Linux: @@ -447,7 +447,7 @@ It is recommended to increase the stacksize limit by using ``ulimit -S -s unlimi .. code-block:: console - export SPACK_SYSTEM_CONFIG_PATH="$PWD/envs/unified-env.mylinux/site" + export SPACK_SYSTEM_CONFIG_PATH="$PWD/site" 4. Find external packages, add to site config's ``packages.yaml``. If an external's bin directory hasn't been added to ``$PATH``, need to prefix command. @@ -498,14 +498,14 @@ It is recommended to increase the stacksize limit by using ``ulimit -S -s unlimi spack config add "packages:fontconfig:variants:+pic" spack config add "packages:pixman:variants:+pic" spack config add "packages:cairo:variants:+pic" - spack config add "packages:libffi:version:[3.3]" - spack config add "packages:flex:version:[2.6.4]" + spack config add "packages:libffi:version:['3.3']" + spack config add "packages:flex:version:['2.6.4']" 9. If you have manually installed lmod, you will need to update the site module configuration to use lmod instead of tcl. Skip this step if you followed the Ubuntu or Red Hat instructions above. .. code-block:: console - sed -i 's/tcl/lmod/g' envs/unified-env.mylinux/site/modules.yaml + sed -i 's/tcl/lmod/g' site/modules.yaml 10. If applicable (depends on the environment), edit the main config file for the environment and adjust the compiler matrix to match the compilers for Linux, as above: @@ -514,13 +514,13 @@ It is recommended to increase the stacksize limit by using ``ulimit -S -s unlimi definitions: - compilers: ['%gcc'] -11. Edit site config files and common config files, for example to remove duplicate versions of external packages that are unwanted, add specs in ``envs/unified-env.mylinux/spack.yaml``, etc. +11. Edit site config files and common config files, for example to remove duplicate versions of external packages that are unwanted, add specs in ``spack.yaml``, etc. .. code-block:: console - vi envs/unified-env.mylinux/spack.yaml - vi envs/unified-env.mylinux/common/*.yaml - vi envs/unified-env.mylinux/site/*.yaml + vi spack.yaml + vi common/*.yaml + vi site/*.yaml 12. Process the specs and install @@ -532,8 +532,8 @@ See the :ref:`documentation ` for usage information including .. code-block:: console spack concretize 2>&1 | tee log.concretize - util/show_duplicate_packages.py -d [-c] log.concretize - spack install [--verbose] [--fail-fast] + ${SPACK_STACK_DIR}/util/show_duplicate_packages.py -d [-c] log.concretize + spack install [--verbose] [--fail-fast] 2>&1 | tee log.install 13. Create tcl module files (replace ``tcl`` with ``lmod`` if you have manually installed lmod) @@ -547,4 +547,4 @@ See the :ref:`documentation ` for usage information including spack stack setup-meta-modules -15. You now have a spack-stack environment that can be accessed by running ``module use ./envs/unified-env.mylinux/install/modulefiles/Core``. The modules defined here can be loaded to build and run code as described in :numref:`Section %s `. +15. You now have a spack-stack environment that can be accessed by running ``module use ${SPACK_STACK_DIR}/envs/unified-env.mylinux/install/modulefiles/Core``. The modules defined here can be loaded to build and run code as described in :numref:`Section %s `. diff --git a/doc/source/PreConfiguredSites.rst b/doc/source/PreConfiguredSites.rst index ee620493c..e50bcec06 100644 --- a/doc/source/PreConfiguredSites.rst +++ b/doc/source/PreConfiguredSites.rst @@ -20,31 +20,31 @@ Ready-to-use spack-stack 1.5.1 installations are available on the following, ful +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | | Hercules^* | GCC, Intel | ``/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.5.1/envs/unified-env`` | Cam Book / Dom Heinzeller | | MSU +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| | Orion | GCC, Intel | ``/work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.0/envs/unified-env`` | Cam Book / Dom Heinzeller | +| | Orion | GCC, Intel | ``/work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/envs/unified-env`` | Cam Book / Dom Heinzeller | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | NASA | Discover | GCC, Intel | ``/gpfsm/dswdev/jcsda/spack-stack/spack-stack-1.5.1/envs/unified-env`` | Dom Heinzeller / ??? | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| | Casper | Intel | ``/glade/work/epicufsrt/contrib/spack-stack/casper/spack-stack-1.5.0/envs/unified-env`` | Dom Heinzeller / ??? | +| | Casper | GCC | ``/glade/work/epicufsrt/contrib/spack-stack/casper/spack-stack-1.5.1/envs/unified-env`` | Dom Heinzeller / ??? | | +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | NCAR-Wyoming | Cheyenne | GCC, Intel | ``/glade/work/epicufsrt/contrib/spack-stack/cheyenne/spack-stack-1.5.0/envs/{unified-env,ufs-env}`` | Cam Book / Dom Heinzeller | | +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | | Derecho | GCC, Intel | ``/glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.5.1/envs/unified-env`` | Dom Heinzeller / Mark Potts | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| NOAA (NCEP) | Acorn | Intel | ``/lfs/h1/emc/nceplibs/noscrub/spack-stack/spack-stack-1.5.0/envs/unified-env`` | Hang Lei / Alex Richert | +| NOAA (NCEP) | Acorn | Intel | ``/lfs/h1/emc/nceplibs/noscrub/spack-stack/spack-stack-1.5.1/envs/unified-env`` | Hang Lei / Alex Richert | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| | Gaea C4 | Intel | ``/lustre/f2/dev/wpo/role.epic/contrib/spack-stack/c4/spack-stack-1.5.0/envs/unified-env`` | Dom Heinzeller / Cam Book | +| | Gaea C4 | Intel | ``/lustre/f2/dev/wpo/role.epic/contrib/spack-stack/c4/spack-stack-1.5.1/envs/unified-env`` | Dom Heinzeller / Cam Book | | +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | | Gaea C5 | Intel | ``/lustre/f2/dev/wpo/role.epic/contrib/spack-stack/c5/spack-stack-1.5.1/envs/unified-env`` | Dom Heinzeller / Cam Book | | NOAA (RDHPCS) +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| | Hera^** | GCC, Intel | ``/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.5.0/envs/unified-env-noavx512`` | Mark Potts / Dom Heinzeller | +| | Hera | GCC, Intel | ``/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.5.1/envs/unified-env`` | Mark Potts / Dom Heinzeller | | +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| | Jet^** | GCC, Intel | ``/mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.5.0/envs/unified-env`` | Cam Book / Dom Heinzeller | +| | Jet | GCC, Intel | ``/mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.5.1/envs/unified-env`` | Cam Book / Dom Heinzeller | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | | Narwhal | Intel | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.5.1/envs/unified-env-intel-2021.4.0`` | Dom Heinzeller / Sarah King | | +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | | Narwhal | GCC | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.5.1/envs/unified-env-gcc-10.3.0`` | Dom Heinzeller / Sarah King | | U.S. Navy (HPCMP) +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| | Nautilus | Intel^* | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.5.0/envs/unified-env`` | Dom Heinzeller / Sarah King | +| | Nautilus | Intel^* | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.5.1/envs/unified-env`` | Dom Heinzeller / Sarah King | | +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | | Nautilus | AOCC | *currently not supported* | Dom Heinzeller / Sarah King | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ @@ -54,17 +54,15 @@ Ready-to-use spack-stack 1.5.1 installations are available on the following, ful +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | **Cloud platforms** | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| | AMI Red Hat 8 | GCC | ``/home/ec2-user/spack-stack/spack-stack-1.5.0/envs/unified-env`` | Dom Heinzeller / ??? | +| | AMI Red Hat 8 | GCC | ``/home/ec2-user/spack-stack/spack-stack-1.5.1/envs/unified-env`` | Dom Heinzeller / ??? | + Amazon Web Services +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | | Parallelcluster JCSDA R&D | Intel | ``/mnt/experiments-efs/skylab-v7/spack-stack-1.5.1/envs/unified-env`` | Dom Heinzeller / ??? | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ -| NOAA (RDHPCS) | RDHPCS Cloud (Parallel Works)^** | Intel | ``/contrib/spack-stack/spack-stack-1.5.0/envs/unified-env`` | Mark Potts / Cam Book / Dom H | +| NOAA (RDHPCS) | RDHPCS Cloud (Parallel Works) | Intel | ``/contrib/spack-stack/spack-stack-1.5.1/envs/unified-env`` | Mark Potts / Cam Book / Dom H | +---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ ^* Uses a different ``wgrib2`` version 3.1.1 than the default 2.0.8. -^** Has an additional package ``yafyaml`` installed in the unified environment. - For questions or problems, please consult the known issues in :numref:`Section %s `, the currently open GitHub `issues `_ and `discussions `_ first. .. _Preconfigured_Sites_Tier1: @@ -89,21 +87,21 @@ The following is required for building new spack environments and for using spac module load ecflow/5.8.4 module load mysql/8.0.31 -For ``spack-stack-1.5.0`` with Intel, load the following modules after loading miniconda and ecflow: +For ``spack-stack-1.5.1`` with Intel, load the following modules after loading miniconda and ecflow: .. code-block:: console - module use /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core + module use /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-intel/2022.0.2 module load stack-intel-oneapi-mpi/2021.5.1 module load stack-python/3.10.8 module available -For ``spack-stack-1.5.0`` with GNU, load the following modules after loading miniconda and ecflow: +For ``spack-stack-1.5.1`` with GNU, load the following modules after loading miniconda and ecflow: .. code-block:: console - module use /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core + module use /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-gcc/10.2.0 module load stack-openmpi/4.0.4 module load stack-python/3.10.8 @@ -137,7 +135,7 @@ For ``spack-stack-1.5.1`` with GNU, load the following modules after loading mys .. code-block:: console module use /work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core - module load stack-gcc/11.3.1 + module load stack-gcc/12.2.0 module load stack-mvapich2/2.3.7 module load stack-python/3.10.8 module available @@ -263,11 +261,11 @@ With Intel, the following is required for building new spack environments and fo module load ecflow/5.8.4 module load mysql/8.0.31 -For ``spack-stack-1.5.0`` with Intel, load the following modules after loading the above modules. +For ``spack-stack-1.5.1`` with Intel, load the following modules after loading the above modules. .. code-block:: console - module use /p/app/projects/NEPTUNE/spack-stack/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core + module use /p/app/projects/NEPTUNE/spack-stack/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-intel/2021.5.0 module load stack-openmpi/4.1.5rc2 module load stack-python/3.10.8 @@ -290,7 +288,7 @@ With AMD clang/flang (aocc), the following is required for building new spack en .. note:: - ``spack-stack-1.5.0`` is not yet supported with the Arm clang/flang compilers. Use Intel instead. + ``spack-stack-1.5.1`` is not yet supported with the Arm clang/flang compilers. Use Intel instead. .. note:: @@ -307,19 +305,21 @@ The following is required for building new spack environments and for using spac .. code-block:: console module purge + # ignore that the sticky module ncarenv/... is not unloaded export LMOD_TMOD_FIND_FIRST=yes + module load ncarenv/23.10 module use /glade/work/epicufsrt/contrib/spack-stack/casper/modulefiles - module load python/3.7.9 module load ecflow/5.8.4 module load mysql/8.0.31 -For ``spack-stack-1.5.0`` with Intel, load the following modules after loading miniconda and ecflow. +For ``spack-stack-1.5.1`` with GNU, load the following modules after loading the above modules. .. code-block:: console - module use /glade/work/epicufsrt/contrib/spack-stack/casper/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core - module load stack-intel/2022.0.2 - module load stack-intel-oneapi-mpi/2021.5.1 + module use /glade/work/epicufsrt/contrib/spack-stack/casper/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core + + module load stack-gcc/12.2.0 + module load stack-openmpi/4.1.6 module load stack-python/3.10.8 module available @@ -410,7 +410,7 @@ For ``spack-stack-1.5.1`` with GNU, load the following modules after loading ecf NOAA Acorn (WCOSS2 test system) ------------------------------- -For spack-stack-1.5.0, the meta modules are in ``/lfs/h1/emc/nceplibs/noscrub/spack-stack/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core``. +For spack-stack-1.5.1, the meta modules are in ``/lfs/h1/emc/nceplibs/noscrub/spack-stack/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core``. On WCOSS2 OpenSUSE sets ``CONFIG_SITE`` which causes libraries to be installed in ``lib64``, breaking the ``lib`` assumption made by some packages. Therefore, ``CONFIG_SITE`` should be set to empty in ``compilers.yaml``. Also, don't use ``module purge`` on Acorn! @@ -444,11 +444,11 @@ The following is required for building new spack environments and for using spac module load mysql/8.0.31 module load git-lfs/2.4.1 -For ``spack-stack-1.5.0`` with Intel, proceed with loading the following modules: +For ``spack-stack-1.5.1`` with Intel, proceed with loading the following modules: .. code-block:: console - module use /contrib/spack-stack/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core + module use /contrib/spack-stack/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-intel/2021.3.0 module load stack-intel-oneapi-mpi/2021.3.0 module load stack-python/3.10.8 @@ -473,11 +473,11 @@ The following is required for building new spack environments and for using spac module load ecflow/5.8.4 module load mysql/8.0.31 -For ``spack-stack-1.5.0`` with Intel, load the following modules after loading miniconda and ecflow: +For ``spack-stack-1.5.1`` with Intel, load the following modules after loading miniconda and ecflow: .. code-block:: console - module use /lustre/f2/dev/wpo/role.epic/contrib/spack-stack/c4/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core + module use /lustre/f2/dev/wpo/role.epic/contrib/spack-stack/c4/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-intel/2022.0.2 module load stack-cray-mpich/7.7.20 module load stack-python/3.10.8 @@ -545,21 +545,21 @@ The following is required for building new spack environments and for using spac module load ecflow/5.5.3 module load mysql/8.0.31 -For ``spack-stack-1.5.0`` with Intel, load the following modules after loading miniconda and ecflow: +For ``spack-stack-1.5.1`` with Intel, load the following modules after loading miniconda and ecflow: .. code-block:: console - module use /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.5.0/envs/unified-env-noavx512/install/modulefiles/Core + module use /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-intel/2021.5.0 module load stack-intel-oneapi-mpi/2021.5.1 module load stack-python/3.10.8 module available -For ``spack-stack-1.5.0`` with GNU, load the following modules after loading miniconda and ecflow: +For ``spack-stack-1.5.1`` with GNU, load the following modules after loading miniconda and ecflow: .. code-block:: console - module use /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.5.0/envs/unified-env-noavx512/install/modulefiles/Core + module use /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-gcc/9.2.0 module load stack-openmpi/4.1.5 module load stack-python/3.10.8 @@ -584,28 +584,26 @@ The following is required for building new spack environments and for using spac module use /lfs4/HFIP/hfv3gfs/role.epic/modulefiles module load mysql/8.0.31 -For ``spack-stack-1.5.0`` with Intel, load the following modules after loading miniconda and ecflow: +For ``spack-stack-1.5.1`` with Intel, load the following modules after loading miniconda and ecflow: .. code-block:: console - module use /mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core + module use /mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-intel/2021.5.0 module load stack-intel-oneapi-mpi/2021.5.1 module load stack-python/3.10.8 module available -For ``spack-stack-1.5.0`` with GNU, load the following modules after loading miniconda and ecflow: +For ``spack-stack-1.5.1`` with GNU, load the following modules after loading miniconda and ecflow: .. code-block:: console - module use /mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core + module use /mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-gcc/9.2.0 module load stack-openmpi/3.1.4 module load stack-python/3.10.8 module available -Note that the ``spack-stack-1.5.0`` unified environment on Hera has an additional package ``yafyaml`` installed that does not exist in the default 1.5.0 installation. - ------------------------------ UW (Univ. of Wisconsin) S4 ------------------------------ @@ -675,15 +673,15 @@ For ``spack-stack-1.5.1`` with GNU on the JCSDA R&D cluster (``hpc6a.48xlarge`` Amazon Web Services Red Hat 8 ----------------------------- -Use a c6i.4xlarge instance or larger if running out of memory with AMI "skylab-6.0.0-redhat8" (ami-059d445a90ad8b792 in region us-east-1, ami-01fcf5d75ced5a046 in region us-east-2). +Use a c6i.4xlarge instance or larger if running out of memory with AMI "skylab-6.1.0-redhat8" (ami-06497c2e0f2ded6cf in region us-east-1, ami-0b1ce08e2fd42333b in region us-east-2). -For ``spack-stack-1.5.0``, run: +For ``spack-stack-1.5.1``, run: .. code-block:: console ulimit -s unlimited scl enable gcc-toolset-11 bash - module use /home/ec2-user/spack-stack/spack-stack-1.5.0/envs/unified-env/install/modulefiles/Core + module use /home/ec2-user/spack-stack/spack-stack-1.5.1/envs/unified-env/install/modulefiles/Core module load stack-gcc/11.2.1 module load stack-openmpi/4.1.5 module load stack-python/3.10.8 @@ -702,7 +700,7 @@ TACC Frontera ------------------------------ .. note:: - ``spack-stack-1.5.0`` is currently not supported on this platform. + ``spack-stack-1.5.1`` is currently not supported on this platform. The following is required for building new spack environments and for using spack to build and run software. diff --git a/spack b/spack index 12690f08a..b106aa8c3 160000 --- a/spack +++ b/spack @@ -1 +1 @@ -Subproject commit 12690f08aa5df741d1fc9a164b6422d449277adf +Subproject commit b106aa8c3491e5f4d7a9a81f782a0d725bbc2e11