-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update changes from tier1 machines for 1.8.0 release #1310
Merged
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
4f1534a
start branch
RatkoVasic-NOAA e72089f
Update hera mirrors.yaml
RatkoVasic-NOAA fdb48ed
Update jet mirrors.yaml
RatkoVasic-NOAA 81bd9d5
Remove gettext from jet packages.yaml
RatkoVasic-NOAA d51521d
Set prefix for intel-oneapi-mpi Jet packages_intel.yaml
RatkoVasic-NOAA 18f00c1
Add gcc to Gaea-C5 compilers.yaml
RatkoVasic-NOAA 87e69f1
Add gcc to gaea-c6 compilers.yaml
RatkoVasic-NOAA ca5ac1d
Update noaa-cloud config files
RatkoVasic-NOAA 64b6010
Remove gettext from packages.yaml in noaa-aws
RatkoVasic-NOAA f4b37c7
Update README.md file for noaa-aws 1.8.0 installation.
RatkoVasic-NOAA e913c15
Checkout correct branch while cloning.
RatkoVasic-NOAA 75b1cfe
chmod 755 to 777
RatkoVasic-NOAA bd69eae
Merge branch 'release/1.8.0' into 1.8.0-fixes
RatkoVasic-NOAA 405dcbf
Use old mirror site
RatkoVasic-NOAA 39aa23d
Update documentation of how to install spack-stack on Tier 1 machines.
RatkoVasic-NOAA ff28e3c
Update doc/source/PreConfiguredSites.rst
RatkoVasic-NOAA 23695e5
Update doc/source/PreConfiguredSites.rst
RatkoVasic-NOAA 32d30aa
Update doc/source/PreConfiguredSites.rst
RatkoVasic-NOAA ac1f9df
Update doc/source/PreConfiguredSites.rst
RatkoVasic-NOAA 5ae1a92
Update doc/source/PreConfiguredSites.rst
RatkoVasic-NOAA 867f8e7
Update doc/source/PreConfiguredSites.rst
RatkoVasic-NOAA 530d106
Update Jet mirrors path
RatkoVasic-NOAA 0ca6ac3
Update mirrors.yaml
RatkoVasic-NOAA 8f39062
Apply changes for NOAA-cloud machines for spack-stack-1.8.0
RatkoVasic-NOAA 41e55e9
Apply suggestions from code review
climbfuji b0ef365
Merge branch 'release/1.8.0' into 1.8.0-fixes
climbfuji ff269c0
Update config files for NOAA-GCP
642ee98
Update config files for NOAA cloud
RatkoVasic-NOAA File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,168 +1,78 @@ | ||
| # Provisiong ParallelWorks Azure clusters | ||
|
|
||
| ## Steps to perform when provisioning a cluster for the very first time | ||
| ## Steps to perform before installing spack-stack version 1.8.0 | ||
|
|
||
| This section provides step by step instructions for installing the basic packages (OS packages, external packages) for spack-stack. Following these steps ensures that the site configuration files in `configs/sites/noaa-aws` work out of the box. | ||
|
|
||
| ### Basic system packages | ||
| ``` | ||
| sudo su | ||
| sudo su - | ||
| chmod 777 /contrib | ||
|
|
||
| # The following three commands were necessary to fix failures accessing these repos | ||
| yum-config-manager --disable intel-clck-2019-repo | ||
| yum-config-manager --disable intel-hpc-platform | ||
| yum-config-manager --disable intelpython | ||
|
|
||
| yum install -y qt5-qtbase-devel | ||
| yum install -y qt5-qtsvg-devel | ||
| yum install -y xorg-x11-xauth | ||
| yum install -y xorg-x11-apps | ||
| yum install -y perl-IPC-Cmd | ||
| yum install -y gettext-devel | ||
| yum install -y ncurses-devel | ||
| yum install -y ncurses-static | ||
| yum install -y m4 | ||
| exit | ||
| ``` | ||
| Create a script that can be added to the cluster resource config so that these packages get installed automatically when provisioning new clusters later: | ||
| ``` | ||
| mkdir -p /contrib/admin | ||
| cat <<EOF > /contrib/admin/basic_setup.sh | ||
| #!/bin/bash | ||
|
|
||
| chmod 777 /contrib | ||
| yum install -y qt5-qtbase-devel | ||
| yum install -y qt5-qtsvg-devel | ||
| yum install -y xorg-x11-xauth | ||
| yum install -y xorg-x11-apps | ||
| yum install -y perl-IPC-Cmd | ||
| yum install -y gettext-devel | ||
| yum install -y ncurses-devel | ||
| yum install -y ncurses-static | ||
| yum install -y m4 | ||
| EOF | ||
|
|
||
| chmod a+x /contrib/admin/basic_setup.sh | ||
| ``` | ||
| ## Steps to install spack-stack version 1.8.0 | ||
|
|
||
| ### Create a mysql config for local R2D2 use (if applicable) | ||
| ``` | ||
| sudo su | ||
| cat <<EOF > /contrib/admin/my.cnf | ||
| [mysqld] | ||
| datadir=/mysql_local/data | ||
| socket=/mysql_local/data/mysql.sock | ||
| symbolic-links=0 | ||
| default-authentication-plugin=mysql_native_password | ||
| sudo su - | ||
| chmod 777 /contrib | ||
|
|
||
| [mysqld_safe] | ||
| log-error=/mysql_local/log/mariadb.log | ||
| pid-file=/mysql_local/run/mariadb.pid | ||
| EOF | ||
| chmod 644 /contrib/admin/my.cnf | ||
| exit | ||
| ``` | ||
| module purge | ||
| module unuse /opt/cray/craype/default/modulefiles | ||
| module unuse /opt/cray/modulefiles | ||
| module load gnu | ||
| module load intel/2023.2.0 | ||
| module load impi/2023.2.0 | ||
| module unload gnu | ||
|
|
||
| cd /contrib/spack-stack-rocky8/ | ||
| git clone --recursive https://github.com/JCSDA/spack-stack -b release/1.8.0 spack-stack-1.8.0 | ||
| cd spack-stack-1.8.0 | ||
| . setup.sh | ||
| spack stack create env --name ue-intel-2021.10.0 --template unified-dev --site noaa-azure --compiler intel | ||
| cd envs/ue-intel-2021.10.0 | ||
| spack env activate . | ||
| spack concretize 2>&1 | tee log.concretize | ||
| spack install --verbose 2>&1 | tee log.install | ||
| spack module lmod refresh -y | ||
| spack stack setup-meta-modules | ||
|
|
||
| ## Steps to install GSI addon | ||
|
|
||
| sudo su - | ||
| chmod 777 /contrib | ||
|
|
||
| ### Build external packages for spack-stack | ||
| ``` | ||
| mkdir -p /contrib/spack-stack | ||
| mkdir /contrib/spack-stack/modulefiles | ||
| cd /contrib/spack-stack/ | ||
| cd /contrib/spack-stack-rocky8/spack-stack-1.8.0 | ||
| . setup.sh | ||
| spack stack create env --name gsi-intel-2021.10.0 --template gsi-addon-dev --site noaa-azure --upstream /contrib/spack-stack-rocky8/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install --compiler intel | ||
| cd envs/gsi-intel-2021.10.0 | ||
| spack concretize 2>&1 | tee log.concretize | ||
| spack install --verbose 2>&1 | tee log.install | ||
| spack module lmod refresh --upstream-modules | ||
| spack stack setup-meta-modules | ||
|
|
||
| mkdir -p git-lfs-2.4.1/src | ||
| cd git-lfs-2.4.1/src | ||
| wget http://mirror.centos.org/centos/7/sclo/x86_64/rh/Packages/r/rh-git218-git-lfs-2.4.1-3.el7.x86_64.rpm | ||
| rpm2cpio rh-git218-git-lfs-2.4.1-3.el7.x86_64.rpm | cpio -idmv | ||
| mv opt/rh/rh-git218/root/usr/* .. | ||
| rm -fr opt | ||
| cd /contrib/spack-stack/modulefiles | ||
| mkdir git-lfs | ||
| # Create the modulefile from the template in doc/modulefile_templates | ||
| ## Steps to install external applications (MySQL, ecFlow) | ||
|
|
||
| cd /contrib/spack-stack/ | ||
| mkdir -p mysql-8.0.31/src | ||
| cd mysql-8.0.31/src | ||
| ldd --version | ||
| MySQL: | ||
| mkdir -p /contrib/spack-stack-rocky8/mysql-8.0.31/src | ||
| cd /contrib/spack-stack-rocky8/mysql-8.0.31/src | ||
| wget https://dev.mysql.com/get/Downloads/MySQL-8.0/mysql-8.0.31-linux-glibc2.17-x86_64-minimal.tar.xz | ||
| tar -xvf mysql-8.0.31-linux-glibc2.17-x86_64-minimal.tar.xz | ||
| mv mysql-8.0.31-linux-glibc2.17-x86_64-minimal/* .. | ||
| rmdir mysql-8.0.31-linux-glibc2.17-x86_64-minimal | ||
| cd /contrib/spack-stack/modulefiles | ||
| mkdir mysql | ||
| # Create the modulefile from the template in doc/modulefile_templates | ||
|
|
||
| cd /contrib/spack-stack/ | ||
| mkdir -p cmake-3.27.2/src | ||
| cd cmake-3.27.2/src | ||
| wget https://github.com/Kitware/CMake/releases/download/v3.27.2/cmake-3.27.2-linux-x86_64.tar.gz | ||
| tar -xvzf cmake-3.27.2-linux-x86_64.tar.gz | ||
| mv cmake-3.27.2-linux-x86_64/* .. | ||
| rmdir cmake-3.27.2-linux-x86_64 | ||
| cd /contrib/spack-stack/modulefiles | ||
| mkdir cmake | ||
| # Create the modulefile from the template in doc/modulefile_templates | ||
|
|
||
| # Set up basic modules for building the external ecflow package | ||
| module unuse /opt/cray/craype/default/modulefiles | ||
| module unuse /opt/cray/modulefiles | ||
|
|
||
| module purge | ||
| module load gnu/9.2.0 | ||
| module use /contrib/spack-stack/modulefiles | ||
| module load cmake/3.27.2 | ||
|
|
||
| mkdir -p /contrib/spack-stack/ecflow-5.8.4/src | ||
| cd /contrib/spack-stack/ecflow-5.8.4/src | ||
| ecFlow: | ||
| mkdir -p /contrib/spack-stack-rocky8/ecflow-5.8.4/src | ||
| cd /contrib/spack-stack-rocky8/ecflow-5.8.4/src | ||
| wget https://confluence.ecmwf.int/download/attachments/8650755/ecFlow-5.8.4-Source.tar.gz?api=v2 | ||
| wget https://boostorg.jfrog.io/artifactory/main/release/1.78.0/source/boost_1_78_0.tar.gz | ||
| mv ecFlow-5.8.4-Source.tar.gz\?api\=v2 ecFlow-5.8.4-Source.tar.gz | ||
| tar -xvzf boost_1_78_0.tar.gz | ||
| tar -xvzf ecFlow-5.8.4-Source.tar.gz | ||
| export WK=/contrib/spack-stack/ecflow-5.8.4/src/ecFlow-5.8.4-Source | ||
| export BOOST_ROOT=/contrib/spack-stack/ecflow-5.8.4/src/boost_1_78_0 | ||
|
|
||
| # Build static boost (to not interfere with spack-stack boost) | ||
| export WK=/contrib/spack-stack-rocky8/ecflow-5.8.4/src/ecFlow-5.8.4-Source | ||
| export BOOST_ROOT=/contrib/spack-stack-rocky8/ecflow-5.8.4/src/boost_1_78_0 | ||
| cd $BOOST_ROOT | ||
| ./bootstrap.sh 2>&1 | tee bootstrap.log | ||
| $WK/build_scripts/boost_build.sh 2>&1 | tee boost_build.log | ||
|
|
||
| # Build ecFlow | ||
| cd $WK | ||
| mkdir build | ||
| cd build | ||
| cmake .. -DCMAKE_INSTALL_PREFIX=/contrib/spack-stack/ecflow-5.8.4 2>&1 | tee log.cmake | ||
| cmake .. -DPython3_EXECUTABLE=`which python3` -DCMAKE_INSTALL_PREFIX=/contrib/spack-stack-rocky8/ecflow-5.8.4 2>&1 | tee log.cmake | ||
| make -j4 2>&1 | tee log.make | ||
| make install 2>&1 | tee log.install | ||
|
|
||
| cd /contrib/spack-stack/modulefiles | ||
| mkdir ecflow | ||
| # Create the modulefile from the template in doc/modulefile_templates | ||
| ``` | ||
|
|
||
| ## Steps to perform when starting a new cluster | ||
|
|
||
| This should be done automatically, but doesn't hurt to run again just in case: | ||
| ``` | ||
| sudo /contrib/admin/basic_setup.sh | ||
| ``` | ||
| Configure `git` and `aws` command line utilities | ||
| ``` | ||
| module unuse /opt/cray/craype/default/modulefiles | ||
| module unuse /opt/cray/modulefiles | ||
| module use /contrib/spack-stack/modulefiles | ||
| module load cmake/3.27.2 | ||
| module load ecflow/5.8.4 | ||
| module load mysql/8.0.31 | ||
| module load git-lfs/2.4.1 | ||
|
|
||
| git lfs install | ||
| git config --global credential.helper cache # or store | ||
| git config --global user.email "your.name@domain.com" | ||
| git config --global user.name "Your Name" | ||
|
|
||
| mkdir ~/.aws | ||
| # Create ~/.aws/config | ||
| # Create ~/.aws/credentials | ||
| chmod 400 ~/.aws/credentials | ||
| ``` |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why ecflow? I thought we had discussed that we don't need this anymore. But ok, let's leave it in the readme, it won't get used as long as the site config has it removed from the external packages and the qt5 external package added instead (if it isn't there already).