Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(ci): Use retry actions to prevent ci jank #1481

Closed
wants to merge 5 commits into from
Closed

chore(ci): Use retry actions to prevent ci jank #1481

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2fcded2
chore(ci): Use retry actions to prevent ci jank
m2Giles Jul 2, 2024
837e676
chore(ci): prepull image for ISO build
m2Giles Jul 2, 2024
9006e12
chore(ci): Don't use retry for nested actions
m2Giles Jul 2, 2024
16a18df
chore(ci): Remove unneeded chainguard check
m2Giles Jul 2, 2024
c873447
chore(ci): Don't retry kernel signer
m2Giles Jul 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 67 additions & 46 deletions .github/workflows/reusable-build-iso.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,13 +73,16 @@ jobs:

- name: Get Current Fedora Version
id: labels
shell: bash
run: |
set -eo pipefail
kernel_release=$(skopeo inspect docker://ghcr.io/ublue-os/${{ env.IMAGE_NAME }}:${{ inputs.fedora_version }} | jq -r '.Labels["ostree.linux"] | split(".x86_64")[0]')
fedora_version=$(echo $kernel_release | grep -oP 'fc\K[0-9]+')
echo "kernel_release=$kernel_release" >> $GITHUB_OUTPUT
echo "fedora_version=$fedora_version" >> $GITHUB_OUTPUT
uses: Wandalen/wretry.action@v3.5.0
with:
attempt_limit: 3
attempt_delay: 15000
command: |
set -eo pipefail
kernel_release=$(skopeo inspect docker://ghcr.io/ublue-os/${{ env.IMAGE_NAME }}:${{ inputs.fedora_version }} | jq -r '.Labels["ostree.linux"] | split(".x86_64")[0]')
fedora_version=$(echo $kernel_release | grep -oP 'fc\K[0-9]+')
echo "kernel_release=$kernel_release" >> $GITHUB_ENV
echo "fedora_version=$fedora_version" >> $GITHUB_ENV

- name: Set Image Tag
id: generate-tag
Expand Down Expand Up @@ -115,36 +118,48 @@ jobs:
fi
echo "flatpak-dir-shortname=${FLATPAK_DIR_SHORTNAME}" >> $GITHUB_OUTPUT

- name: Pull main and akmods images
uses: Wandalen/wretry.action@v3.5.0
with:
attempt_limit: 3
attempt_delay: 15000
command: |
# pull the base images used for FROM in Containerfile so
# we can retry on that unfortunately common failure case
docker pull "ghcr.io/ublue-os/${{ env.IMAGE_NAME }}:${{ inputs.fedora_version }}"

- name: Determine Flatpak Dependencies
id: flatpak_dependencies
shell: bash
run: |
set -ex
image="ghcr.io/ublue-os/${{ env.IMAGE_NAME }}:${{ inputs.fedora_version }}"
# Make temp space
TEMP_FLATPAK_INSTALL_DIR=$(mktemp -d -p ${{ github.workspace }} flatpak.XXX)
# Get list of refs from directory
FLATPAK_REFS_DIR=${{ github.workspace }}/${{ steps.generate-flatpak-dir-shortname.outputs.flatpak-dir-shortname }}
FLATPAK_REFS_DIR_LIST=$(cat ${FLATPAK_REFS_DIR}/* | tr '\n' ' ' )
# Generate install script
cat << EOF > ${TEMP_FLATPAK_INSTALL_DIR}/script.sh
cat /temp_flatpak_install_dir/script.sh
mkdir -p /flatpak/flatpak /flatpak/triggers
mkdir /var/tmp || true
chmod -R 1777 /var/tmp
flatpak config --system --set languages "*"
flatpak remote-add --system flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install --system -y ${FLATPAK_REFS_DIR_LIST}
ostree refs --repo=\${FLATPAK_SYSTEM_DIR}/repo | grep '^deploy/' | grep -v 'org\.freedesktop\.Platform\.openh264' | sed 's/^deploy\///g' > /output/flatpaks_with_deps
EOF
docker run --rm --privileged \
--entrypoint bash \
-e FLATPAK_SYSTEM_DIR=/flatpak/flatpak \
-e FLATPAK_TRIGGERSDIR=/flatpak/triggers \
--volume ${FLATPAK_REFS_DIR}:/output \
--volume ${TEMP_FLATPAK_INSTALL_DIR}:/temp_flatpak_install_dir \
${image} /temp_flatpak_install_dir/script.sh
docker rmi ${image}
uses: Wandalen/wretry.action@v3.5.0
with:
attempt_limit: 3
attempt_delay: 15000
command: |
set -ex
image="ghcr.io/ublue-os/${{ env.IMAGE_NAME }}:${{ inputs.fedora_version }}"
# Make temp space
TEMP_FLATPAK_INSTALL_DIR=$(mktemp -d -p ${{ github.workspace }} flatpak.XXX)
# Get list of refs from directory
FLATPAK_REFS_DIR=${{ github.workspace }}/${{ steps.generate-flatpak-dir-shortname.outputs.flatpak-dir-shortname }}
FLATPAK_REFS_DIR_LIST=$(cat ${FLATPAK_REFS_DIR}/* | tr '\n' ' ' )
# Generate install script
cat << EOF > ${TEMP_FLATPAK_INSTALL_DIR}/script.sh
cat /temp_flatpak_install_dir/script.sh
mkdir -p /flatpak/flatpak /flatpak/triggers
mkdir /var/tmp || true
chmod -R 1777 /var/tmp
flatpak config --system --set languages "*"
flatpak remote-add --system flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install --system -y ${FLATPAK_REFS_DIR_LIST}
ostree refs --repo=\${FLATPAK_SYSTEM_DIR}/repo | grep '^deploy/' | grep -v 'org\.freedesktop\.Platform\.openh264' | sed 's/^deploy\///g' > /output/flatpaks_with_deps
EOF
docker run --rm --privileged \
--entrypoint bash \
-e FLATPAK_SYSTEM_DIR=/flatpak/flatpak \
-e FLATPAK_TRIGGERSDIR=/flatpak/triggers \
--volume ${FLATPAK_REFS_DIR}:/output \
--volume ${TEMP_FLATPAK_INSTALL_DIR}:/temp_flatpak_install_dir \
${image} /temp_flatpak_install_dir/script.sh

- name: Build ISOs
uses: jasonn3/build-container-installer@v1.2.1
Expand All @@ -154,7 +169,7 @@ jobs:
image_name: ${{ env.IMAGE_NAME }}
image_repo: ghcr.io/ublue-os
variant: ${{ env.VARIANT }}
version: ${{ steps.labels.outputs.fedora_version }}
version: ${{ env.fedora_version }}
image_tag: ${{ steps.generate-tag.outputs.tag }}
secure_boot_key_url: "https://github.com/ublue-os/akmods/raw/main/certs/public_key.der"
enrollment_password: "ublue-os"
Expand Down Expand Up @@ -188,7 +203,7 @@ jobs:

- name: Upload ISOs and Checksum to R2 to Bluefin Bucket
if: github.ref_name == 'main' && inputs.brand_name == 'bluefin'
shell: bash
uses: Wandalen/wretry.action@v3.5.0
env:
RCLONE_CONFIG_R2_TYPE: s3
RCLONE_CONFIG_R2_PROVIDER: Cloudflare
Expand All @@ -197,14 +212,17 @@ jobs:
RCLONE_CONFIG_R2_REGION: auto
RCLONE_CONFIG_R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
SOURCE_DIR: ${{ steps.upload-directory.outputs.iso-upload-dir }}
run: |
sudo apt-get update
sudo apt-get install -y rclone
rclone copy $SOURCE_DIR R2:bluefin
with:
attempt_limit: 3
attempt_delay: 15000
command: |
sudo apt-get update
sudo apt-get install -y rclone
rclone copy $SOURCE_DIR R2:bluefin

- name: Upload ISOs and Checksum to R2 to Aurora Bucket
if: github.ref_name == 'main' && inputs.brand_name == 'aurora'
shell: bash
uses: Wandalen/wretry.action@v3.5.0
env:
RCLONE_CONFIG_R2_TYPE: s3
RCLONE_CONFIG_R2_PROVIDER: Cloudflare
Expand All @@ -213,7 +231,10 @@ jobs:
RCLONE_CONFIG_R2_REGION: auto
RCLONE_CONFIG_R2_ENDPOINT: ${{ secrets.AURORA_R2_ENDPOINT }}
SOURCE_DIR: ${{ steps.upload-directory.outputs.iso-upload-dir }}
run: |
sudo apt-get update
sudo apt-get install -y rclone
rclone copy $SOURCE_DIR R2:aurora-dl
with:
attempt_limit: 3
attempt_delay: 15000
command: |
sudo apt-get update
sudo apt-get install -y rclone
rclone copy $SOURCE_DIR R2:aurora-dl
Loading
Loading