Update changes from tier1 machines for 1.8.0 release #1310

RatkoVasic-NOAA · 2024-09-25T15:54:31Z

Summary

Update changes for tier1 machines for 1.8.0

Testing

Install 1.8.0

Applications affected

All

Systems affected

Tier1

Dependencies

Issue(s) addressed

Part of issue #1278

Checklist

This PR addresses one issue/problem/enhancement, or has a very good reason for not doing so.
These changes have been tested on the affected systems and applications.
All dependency PRs/issues have been resolved and this PR can be merged.

Place mirror files to more friendly space

configs/sites/tier1/hera/mirrors.yaml

climbfuji · 2024-09-27T19:56:38Z

configs/sites/tier1/noaa-aws/README.md

-mkdir -p /contrib/admin
-cat <<EOF > /contrib/admin/basic_setup.sh
-#!/bin/bash
+## Steps to perform to install spack-stack version 1.8.0


what happened to all the instructions on the left - are they now baked into the default image? if not, they need to stay because somebody starting from a new image will need them

This is not needed anymore. All was because ecflow and it's dependences. On NOAA-Cloud spack-stack can now be installed from out-of-box.

That's awesome, thanks for the clarification!

As discussed, please bring the documentation back that is needed to install the qt@5 dependency for ecflow

I'll add part how to install qt5

climbfuji · 2024-09-27T19:57:13Z

configs/sites/tier1/noaa-aws/packages.yaml

      prefix: /usr
  diffutils:
    externals:
-    - spec: diffutils@3.3
+    - spec: diffutils@3.6
      prefix: /usr
  ecflow::


no external ecflow - but need external qt@5

That is now in common/modules.yaml and common/packages.yaml:

qt: require: '@5'

yes but you should configure the external qt5 package so that you don't need to build it (often doesn't work as expected). since you were able to build ecflow outside of spack, the necessary qt 5 libraries must be somewhere on the system

OK, I'm not sure how it worked. I intentionally didn't install:

#yum install -y qt5-qtbase-devel #yum install -y qt5-qtsvg-devel #yum install -y xorg-x11-xauth #yum install -y xorg-x11-apps #yum install -y perl-IPC-Cmd #yum install -y gettext-devel #yum install -y m4

Which we were supposed to install each time we start cloud instance.
Ecflow was installed using qt5 (which will be lost each time we logout).
Are you saying that we have to install qt5 again each time we login (like before)?
ecflow is still there, on permanent place (/contrib/spack-stack-rocky8/ecflow-5.8.4/)

As for m4, gettext, ... they were successfully installed by spack-satck-1.8.0.

I think you need to install those packages every time, but you can easily create a bootstrap script and run that in your parallelworks config

That's OK, but why do we need to install them anyway, spack stack does not depend on them?

qt5 does depend on them, and you need qt5 to build ecflow+ui - you can choose to let spack build all of that, not sure it works though

climbfuji · 2024-09-27T19:57:35Z

configs/sites/tier1/noaa-aws/packages.yaml

      prefix: /usr
  pkg-config:
    externals:
-    - spec: pkg-config@0.27.1
+    - spec: pkg-config@1.4.2
      prefix: /usr
  rsync:


external qt@5 missing

That is now in common/modules.yaml and common/packages.yaml:

qt: require: '@5'

configs/sites/tier1/hera/mirrors.yaml

climbfuji · 2024-10-01T02:14:19Z

configs/sites/tier1/noaa-aws/README.md

-mkdir -p /contrib/admin
-cat <<EOF > /contrib/admin/basic_setup.sh
-#!/bin/bash
+## Steps to perform to install spack-stack version 1.8.0


As discussed, please bring the documentation back that is needed to install the qt@5 dependency for ecflow

doc/source/PreConfiguredSites.rst

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

doc/source/PreConfiguredSites.rst

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

natalie-perlin · 2024-10-01T15:44:25Z

configs/sites/tier1/noaa-aws/README.md

-### Create a mysql config for local R2D2 use (if applicable)
-```
-sudo su
-cat <<EOF > /contrib/admin/my.cnf


We may still need to have /contrib/admin/my.cnf saved. It may not need to be done every time a new space for spack stack is configured, i.e., a new /contrib space is prepared

climbfuji · 2024-10-02T02:28:50Z

configs/sites/tier1/noaa-aws/README.md

+mv mysql-8.0.31-linux-glibc2.17-x86_64-minimal/* ..
+rmdir mysql-8.0.31-linux-glibc2.17-x86_64-minimal
+
+ ecFlow:


Why ecflow? I thought we had discussed that we don't need this anymore. But ok, let's leave it in the readme, it won't get used as long as the site config has it removed from the external packages and the qt5 external package added instead (if it isn't there already).

doc/source/PreConfiguredSites.rst

climbfuji

Instructions for building external ecflow packages are still in the README for all three sites. The site configs still have external ecflow packages, but no external qt5 packages. I'll wait with merging this PR until Wednesday noon but then it will go in as is and the release will be tagged.

RatkoVasic-NOAA · 2024-10-02T03:42:50Z

Instructions for building external ecflow packages are still in the README for all three sites. The site configs still have external ecflow packages, but no external qt5 packages. I'll wait with merging this PR until Wednesday noon but then it will go in as is and the release will be tagged.

@climbfuji I'm finishing NOAA-cloud. GCP is installed. Azure is slower. I'm going to add 2 more commits for these two machines (as we speak).

RatkoVasic-NOAA · 2024-10-02T04:07:05Z

@climbfuji I'm done with NOAA cloud updates. Removed instructions for ecflow as well.

climbfuji · 2024-10-03T00:17:52Z

The three noaa-{aws,gcloud,azure} site configs still have external ecflow packages and no external qt5 packages. I am going to merge this PR nonetheless so that we can tag the release and be done with it.

RatkoVasic-NOAA · 2024-10-03T00:37:47Z

The three noaa-{aws,gcloud,azure} site configs still have external ecflow packages and no external qt5 packages. I am going to merge this PR nonetheless so that we can tag the release and be done with it.

I did that in the meantime. I removed ecflow from packages.yaml, and added external qt5 to it. It was just too late to add it to the repository. BTW, same is with Hera, Jet, Gaeas. I already fixed some of them (Jet, Orion, Hercules) and rest will do tomorrow. Eventually, we can add those to release/1.8.0.

climbfuji · 2024-10-03T01:47:23Z

Fine with me. It won't be in the tagged code, but at least in the release branch. Thanks @RatkoVasic-NOAA !

start branch

4f1534a

RatkoVasic-NOAA self-assigned this Sep 25, 2024

RatkoVasic-NOAA changed the title ~~start branch~~ Update changes from tier1 machines for 1.8.0 release Sep 25, 2024

RatkoVasic-NOAA and others added 11 commits September 25, 2024 18:22

Update hera mirrors.yaml

e72089f

Update jet mirrors.yaml

fdb48ed

Place mirror files to more friendly space

Remove gettext from jet packages.yaml

81bd9d5

Set prefix for intel-oneapi-mpi Jet packages_intel.yaml

d51521d

Add gcc to Gaea-C5 compilers.yaml

18f00c1

Add gcc to gaea-c6 compilers.yaml

87e69f1

Update noaa-cloud config files

ca5ac1d

Remove gettext from packages.yaml in noaa-aws

64b6010

Update README.md file for noaa-aws 1.8.0 installation.

f4b37c7

Checkout correct branch while cloning.

e913c15

chmod 755 to 777

75b1cfe

RatkoVasic-NOAA marked this pull request as ready for review September 27, 2024 19:38

RatkoVasic-NOAA requested review from climbfuji and AlexanderRichert-NOAA September 27, 2024 19:39

Merge branch 'release/1.8.0' into 1.8.0-fixes

bd69eae

AlexanderRichert-NOAA reviewed Sep 27, 2024

View reviewed changes

configs/sites/tier1/hera/mirrors.yaml Outdated Show resolved Hide resolved

AlexanderRichert-NOAA approved these changes Sep 27, 2024

View reviewed changes

climbfuji requested changes Sep 27, 2024

View reviewed changes

RatkoVasic-NOAA added 2 commits September 27, 2024 20:19

Use old mirror site

405dcbf

Update documentation of how to install spack-stack on Tier 1 machines.

39aa23d

climbfuji reviewed Oct 1, 2024

View reviewed changes

RatkoVasic-NOAA and others added 5 commits September 30, 2024 23:46

Update doc/source/PreConfiguredSites.rst

ff28e3c

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

Update doc/source/PreConfiguredSites.rst

23695e5

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

Update doc/source/PreConfiguredSites.rst

32d30aa

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

Update doc/source/PreConfiguredSites.rst

ac1f9df

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

Update doc/source/PreConfiguredSites.rst

5ae1a92

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

climbfuji reviewed Oct 1, 2024

View reviewed changes

doc/source/PreConfiguredSites.rst Outdated Show resolved Hide resolved

RatkoVasic-NOAA and others added 3 commits October 1, 2024 08:13

Update doc/source/PreConfiguredSites.rst

867f8e7

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

Update Jet mirrors path

530d106

Update mirrors.yaml

0ca6ac3

natalie-perlin reviewed Oct 1, 2024

View reviewed changes

Apply changes for NOAA-cloud machines for spack-stack-1.8.0

8f39062

climbfuji reviewed Oct 2, 2024

View reviewed changes

doc/source/PreConfiguredSites.rst Outdated Show resolved Hide resolved

Apply suggestions from code review

41e55e9

climbfuji reviewed Oct 2, 2024

View reviewed changes

Merge branch 'release/1.8.0' into 1.8.0-fixes

b0ef365

root and others added 2 commits October 2, 2024 03:43

Update config files for NOAA-GCP

ff269c0

Update config files for NOAA cloud

642ee98

climbfuji merged commit 801362f into JCSDA:release/1.8.0 Oct 3, 2024
8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update changes from tier1 machines for 1.8.0 release #1310

Update changes from tier1 machines for 1.8.0 release #1310

RatkoVasic-NOAA commented Sep 25, 2024 •

edited

Loading

climbfuji Sep 27, 2024

RatkoVasic-NOAA Sep 27, 2024

climbfuji Sep 27, 2024

climbfuji Oct 1, 2024

RatkoVasic-NOAA Oct 1, 2024

climbfuji Sep 27, 2024

RatkoVasic-NOAA Sep 27, 2024

climbfuji Sep 27, 2024

RatkoVasic-NOAA Sep 27, 2024

climbfuji Sep 27, 2024

RatkoVasic-NOAA Sep 27, 2024

climbfuji Sep 27, 2024

climbfuji Sep 27, 2024

RatkoVasic-NOAA Sep 27, 2024

climbfuji Oct 1, 2024

natalie-perlin Oct 1, 2024

climbfuji Oct 2, 2024

climbfuji left a comment

RatkoVasic-NOAA commented Oct 2, 2024

RatkoVasic-NOAA commented Oct 2, 2024

climbfuji commented Oct 3, 2024

RatkoVasic-NOAA commented Oct 3, 2024

climbfuji commented Oct 3, 2024

Update changes from tier1 machines for 1.8.0 release #1310

Update changes from tier1 machines for 1.8.0 release #1310

Conversation

RatkoVasic-NOAA commented Sep 25, 2024 • edited Loading

Summary

Testing

Applications affected

Systems affected

Dependencies

Issue(s) addressed

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

climbfuji left a comment

Choose a reason for hiding this comment

RatkoVasic-NOAA commented Oct 2, 2024

RatkoVasic-NOAA commented Oct 2, 2024

climbfuji commented Oct 3, 2024

RatkoVasic-NOAA commented Oct 3, 2024

climbfuji commented Oct 3, 2024

RatkoVasic-NOAA commented Sep 25, 2024 •

edited

Loading