Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(cd): enable arm64 for rhel-9 and aws-2023 #11334

Merged
merged 4 commits into from
Aug 3, 2023
Merged

feat(cd): enable arm64 for rhel-9 and aws-2023 #11334

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3fbceff
feat(build): cross compile rpm platforms
fffonion Jul 31, 2023
cf772b1
feat(cd): release rhel9 and aws2023 arm64 artifacts
fffonion Jul 31, 2023
d470208
chore(cd): bump git to 2.41.0 to avoid cache issue
fffonion Aug 1, 2023
36f9887
fix(cd): build package always runs on ubuntu-22.04
fffonion Aug 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 25 additions & 13 deletions .github/matrix-full.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
build-packages:
# label: used to distinguish artifacts for later use
# os: the github actions runner label to pick from
# image: docker image name if the build is running in side a container
# package: package type
# package-type: the nfpm packaging target, //:kong_{package} target; only used when package is rpm
Expand All @@ -9,73 +8,72 @@ build-packages:

# Ubuntu
- label: ubuntu-20.04
os: ubuntu-22.04
image: ubuntu:20.04
package: deb
check-manifest-suite: ubuntu-20.04-amd64
- label: ubuntu-22.04
os: ubuntu-22.04
package: deb
check-manifest-suite: ubuntu-22.04-amd64
- label: ubuntu-22.04-arm64
os: ubuntu-22.04
package: deb
bazel-args: --platforms=//:generic-crossbuild-aarch64
check-manifest-suite: ubuntu-22.04-arm64

# Debian
- label: debian-10
os: ubuntu-22.04
image: debian:10
package: deb
check-manifest-suite: debian-10-amd64
- label: debian-11
os: ubuntu-22.04
image: debian:11
package: deb
check-manifest-suite: debian-11-amd64

# CentOS
- label: centos-7
os: ubuntu-22.04
image: centos:7
package: rpm
package-type: el7
check-manifest-suite: el7-amd64

# RHEL
- label: rhel-7
os: ubuntu-22.04
image: centos:7
package: rpm
package-type: el7
check-manifest-suite: el7-amd64
- label: rhel-8
os: ubuntu-22.04
image: rockylinux:8
package: rpm
package-type: el8
check-manifest-suite: el8-amd64
- label: rhel-9
os: ubuntu-22.04
image: rockylinux:9
package: rpm
package-type: el9
check-manifest-suite: el9-amd64
- label: rhel-9-arm64
package: rpm
package-type: el9
bazel-args: --platforms=//:rhel9-crossbuild-aarch64
check-manifest-suite: el9-arm64

# Amazon Linux
# Amazon Linux
- label: amazonlinux-2
os: ubuntu-22.04
image: amazonlinux:2
package: rpm
package-type: aws2
check-manifest-suite: amazonlinux-2-amd64
- label: amazonlinux-2023
os: ubuntu-22.04
image: amazonlinux:2023
package: rpm
package-type: aws2023
check-manifest-suite: amazonlinux-2023-amd64
- label: amazonlinux-2023-arm64
package: rpm
package-type: aws2023
bazel-args: --platforms=//:aws2023-crossbuild-aarch64
check-manifest-suite: amazonlinux-2023-arm64

build-images:
# Only build images for the latest version of each major release.
Expand Down Expand Up @@ -107,6 +105,8 @@ build-images:
package: rpm
rpm_platform: el9
artifact-from: rhel-9
artifact-from-alt: rhel-9-arm64
docker-platforms: linux/amd64, linux/arm64

smoke-tests:
- label: ubuntu
Expand Down Expand Up @@ -180,6 +180,12 @@ release-packages:
artifact-version: 9
artifact-type: rhel
artifact: kong.el9.amd64.rpm
- label: rhel-9-arm64
package: rpm
artifact-from: rhel-9-arm64
artifact-version: 9
artifact-type: rhel
artifact: kong.el9.arm64.rpm

# Amazon Linux
- label: amazonlinux-2
zhongweiy marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -194,6 +200,12 @@ release-packages:
artifact-version: 2023
artifact-type: amazonlinux
artifact: kong.aws2023.amd64.rpm
- label: amazonlinux-2023-arm64
package: rpm
artifact-from: amazonlinux-2023-arm64
artifact-version: 2023
artifact-type: amazonlinux
artifact: kong.aws2023.arm64.rpm

release-images:
- label: ubuntu
Expand Down
20 changes: 10 additions & 10 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,14 +102,14 @@ jobs:
matrix:
include: "${{ fromJSON(needs.metadata.outputs.matrix)['build-packages'] }}"

runs-on: ${{ matrix.os }}
runs-on: ubuntu-22.04
container:
image: ${{ matrix.image }}
options: --privileged

steps:
- name: Early Rpm Setup
if: matrix.package == 'rpm'
if: matrix.package == 'rpm' && matrix.image != ''
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From line 105, it seems run only on Ubuntu. Then is this matrix.package == 'rpm' check valid?

Copy link
Contributor Author

@fffonion fffonion Aug 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhongweiy currently, we run following builds a on bare ubuntu runner (VM):

  • ubuntu-22.04 builds x86 + arm64 (crossbuild)
  • rhel9, aws2023 arm64 (crossbuild) <= this PR was doing this

the following are ran in a container inside a ubuntu runner VM:

  • all rpm platforms (rhel, centos, aws etc) in x86
  • debian x86 + arm64
  • ubuntu 20.04 x86

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks for the explanation!

run: |
# tar/gzip is needed to restore git cache (if available)
yum install -y tar gzip which file zlib-devel
Expand All @@ -123,15 +123,15 @@ jobs:

- name: Cache Git
id: cache-git
if: matrix.package == 'rpm' || matrix.image == 'debian:10'
if: (matrix.package == 'rpm' || matrix.image == 'debian:10') && matrix.image != ''
uses: actions/cache@v3
with:
path: /usr/local/git
key: ${{ matrix.label }}-git-2.30.0
key: ${{ matrix.label }}-git-2.41.0

# el-7,8, amazonlinux-2,2023, debian-10 doesn't have git 2.18+, so we need to install it manually
- name: Install newer Git
if: (matrix.package == 'rpm' || matrix.image == 'debian:10') && steps.cache-git.outputs.cache-hit != 'true'
if: (matrix.package == 'rpm' || matrix.image == 'debian:10') && matrix.image != '' && steps.cache-git.outputs.cache-hit != 'true'
run: |
if which apt 2>/dev/null; then
apt update
Expand All @@ -141,16 +141,16 @@ jobs:
yum groupinstall -y 'Development Tools'
yum install -y wget zlib-devel openssl-devel curl-devel expat-devel gettext-devel perl-CPAN perl-devel
fi
wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.30.0.tar.gz
tar xf git-2.30.0.tar.gz
cd git-2.30.0
wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.41.0.tar.gz
tar xf git-2.41.0.tar.gz
cd git-2.41.0
make configure
./configure --prefix=/usr/local/git
make -j$(nproc)
make install

- name: Add Git to PATH
if: matrix.package == 'rpm' || matrix.image == 'debian:10'
if: (matrix.package == 'rpm' || matrix.image == 'debian:10') && matrix.image != ''
run: |
echo "/usr/local/git/bin" >> $GITHUB_PATH

Expand Down Expand Up @@ -210,7 +210,7 @@ jobs:
sudo apt-get install crossbuild-essential-arm64 -y

- name: Install Rpm Dependencies
if: matrix.package == 'rpm'
if: matrix.package == 'rpm' && matrix.image != ''
run: |
yum groupinstall -y 'Development Tools'
dnf config-manager --set-enabled powertools || true # enable devel packages on rockylinux:8
Expand Down
41 changes: 20 additions & 21 deletions BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
load("@bazel_skylib//rules:common_settings.bzl", "bool_flag", "string_flag")
load("//build/nfpm:rules.bzl", "nfpm_pkg")
load("@bazel_skylib//lib:selects.bzl", "selects")
load("//build/nfpm:rules.bzl", "nfpm_pkg")
load("//build/toolchain:managed_toolchain.bzl", "aarch64_glibc_distros")

filegroup(
name = "distribution_srcs",
Expand Down Expand Up @@ -335,21 +336,6 @@ selects.config_setting_group(

##### constraints, platforms and config_settings for cross-compile

constraint_setting(name = "libc_version")

constraint_value(
# use whatever glibc version provided by the system installed cross toolchain
name = "glibc_system",
constraint_setting = ":libc_version",
visibility = ["//visibility:public"],
)

constraint_value(
name = "musl",
constraint_setting = ":libc_version",
visibility = ["//visibility:public"],
)

constraint_setting(name = "cross_build_setting")

constraint_value(
Expand All @@ -363,7 +349,7 @@ platform(
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:x86_64",
":glibc_system",
"//build/platforms/distro:generic",
":cross_build",
],
)
Expand All @@ -373,7 +359,7 @@ platform(
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
":glibc_system",
"//build/platforms/distro:generic",
":cross_build",
],
)
Expand All @@ -389,7 +375,7 @@ platform(
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:x86_64",
":musl",
"//build/platforms/distro:alpine",
":cross_build",
],
)
Expand All @@ -405,11 +391,24 @@ platform(
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
":musl",
"//build/platforms/distro:alpine",
":cross_build",
],
)

[
platform(
name = vendor + "-crossbuild-aarch64",
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
"//build/platforms/distro:" + vendor,
":cross_build",
],
)
for vendor in aarch64_glibc_distros
]

# config_settings define a select() condition based on user-set constraint_values
# see https://bazel.build/docs/configurable-attributes
config_setting(
Expand All @@ -427,7 +426,7 @@ config_setting(
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:x86_64",
":musl",
"//build/platforms/distro:alpine",
":cross_build",
],
visibility = ["//visibility:public"],
Expand Down
18 changes: 2 additions & 16 deletions WORKSPACE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,20 +60,6 @@ load("//build/toolchain:repositories.bzl", "toolchain_repositories")

toolchain_repositories()

register_toolchains("//build/toolchain:local_aarch64-linux-gnu_toolchain")
load("//build/toolchain:managed_toolchain.bzl", "register_all_toolchains")

load("//build/toolchain:managed_toolchain.bzl", "register_managed_toolchain")

register_managed_toolchain(
arch = "x86_64",
gcc_version = "11",
libc = "musl",
vendor = "alpine",
)

register_managed_toolchain(
arch = "aarch64",
gcc_version = "11",
libc = "musl",
vendor = "alpine",
)
register_all_toolchains()
20 changes: 20 additions & 0 deletions build/cross_deps/libxcrypt/BUILD.libxcrypt.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
load("@rules_foreign_cc//foreign_cc:defs.bzl", "configure_make")
load("@bazel_skylib//lib:selects.bzl", "selects")
load("@kong_bindings//:variables.bzl", "KONG_VAR")

filegroup(
Expand All @@ -9,6 +10,17 @@ filegroup(
),
)

selects.config_setting_group(
name = "disable-obsolete-api",
# looks like RHEL is aggressive on migrating to libxcrypt
# set this option if any distro is looking for "libcrypto.so.2"
# instead of "libcrypt.so.1" (i.e. "error while loading shared libraries: libcrypt.so.1")
match_any = [
"@kong//build/platforms/distro:rhel9",
"@kong//build/platforms/distro:aws2023",
],
)

configure_make(
name = "libxcrypt",
configure_command = "configure",
Expand All @@ -21,13 +33,21 @@ configure_make(
"--host=x86_64-linux-musl",
],
"//conditions:default": [],
}) + select({
":disable-obsolete-api": [
"--enable-obsolete-api=no",
],
"//conditions:default": [],
}),
lib_source = ":all_srcs",
# out_lib_dir = "lib",
out_shared_libs = select({
"@platforms//os:macos": [
"libcrypt.1.dylib",
],
":disable-obsolete-api": [
"libcrypt.so.2",
],
"//conditions:default": [
"libcrypt.so.1",
],
Expand Down
Loading
Loading