K8SPSMDB-1213 Add arm64 support for the e2e tests #1735

ptankov · 2024-11-20T21:14:50Z

CHANGE DESCRIPTION

Problem:
We want to support the arm64 architecture for our operator releases. We need to re-work some of the test cases and or the supporting functions in the e2e test cases.

Solution:
The solution is to apply tolerations blocks in the yaml resource definitions on the fly
where needed, e.g.:

tolerations:
  - key: node.kubernetes.io/arch
    operator: Equal
    value: arm64
    effect: NoSchedule

Also, replace old docker images which supported only amd64:
amazon/aws-cli instead of perconalab/awscli
alpine/curl instead of appropriate/curl

we're now using -multi images for:
percona/percona-server-mongodb:7.0.14-8-multi
percona/percona-backup-mongodb:2.7.0-multi
and also the
percona/percona-server-mongodb:4.4-multi
image in cases when we use it for the mongo client.

but in the future hopefully the -multi suffix will disappear.

CHECKLIST

Jira

Is the Jira ticket created and referenced properly?
Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

Is an E2E test/test case added for the new feature/change?
Are unit tests added where appropriate?
Are OpenShift compare files changed for E2E tests (compare/*-oc.yml)?

Config/Logging/Testability

Are all needed new/changed options added to default YAML files?
Are all needed new/changed options added to the Helm Chart?
Did we add proper logging messages for operator actions?
Did we ensure compatibility with the previous version or cluster upgrade process?
Does the change support oldest and newest supported MongoDB version?
Does the change support oldest and newest supported Kubernetes version?

Fix tests for 1.18.0 release

…cture

…ied - 02

…ied - 03

…ied - 04

…ied - 05

… function

…n up apply_cluster logic

e2e-tests/upgrade-sharded/run

e2e-tests/upgrade/run

e2e-tests/storage/run

…ackup checks

e2e-tests/default-cr/run

… also need to rename compare files in addition to update of run file content

…ations

…ation

…healing tests

cw-rbac.yaml.1

cw-rbac.yaml

cw-rbac.yaml.2

e2e-tests/data-sharded/run

e2e-tests/default-cr/run

it-percona-cla · 2024-12-04T04:13:54Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ ptankov
❌ eleo007
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

…tecture

…godb-operator into K8SPSMDB-1213

e2e-tests/default-cr/run

e2e-tests/functions

…ble to satisfy reviewdog

… satisfy reviewdog

e2e-tests/default-cr/run

e2e-tests/ldap-tls/run

e2e-tests/balancer/run

e2e-tests/functions

e2e-tests/serviceless-external-nodes/run

github-actions · 2024-12-20T10:57:29Z

e2e-tests/functions

-		| yq eval '(.spec | select(has("initImage"))).initImage = "'"$IMAGE"'"' \
-		| yq eval '(.spec | select(has("backup"))).backup.image = "'"$IMAGE_BACKUP"'"' \
-		| yq eval '.spec.upgradeOptions.apply="Never"'
+    local temp_cr="$(mktemp)"


[shfmt] _{reported by reviewdog 🐶}

Suggested change

local temp_cr="$(mktemp)"

local temp_cr="$(mktemp)"

github-actions · 2024-12-20T10:57:29Z

e2e-tests/functions

+		(.spec | select(has("pmm"))).pmm.image = "'"$IMAGE_PMM_CLIENT"'" |
+		(.spec | select(has("initImage"))).initImage = "'"$IMAGE"'" |
+		(.spec | select(has("backup"))).backup.image = "'"$IMAGE_BACKUP"'" |
+		.spec.upgradeOptions.apply="Never"' "$1" > $temp_cr


[shfmt] _{reported by reviewdog 🐶}

Suggested change

.spec.upgradeOptions.apply="Never"' "$1" > $temp_cr

.spec.upgradeOptions.apply="Never"' "$1" >$temp_cr

github-actions · 2024-12-20T10:57:29Z

e2e-tests/functions

+		.spec.upgradeOptions.apply="Never"' "$1" > $temp_cr
+
+	if [[ $ARCH == "arm64" ]]; then
+        yq eval '.spec.replsets[].tolerations += '"$TOLERATIONS_ARM64"' |


[shfmt] _{reported by reviewdog 🐶}

Suggested change

yq eval '.spec.replsets[].tolerations += '"$TOLERATIONS_ARM64"' |

yq eval '.spec.replsets[].tolerations += '"$TOLERATIONS_ARM64"' |

github-actions · 2024-12-20T10:57:30Z

e2e-tests/functions

+
+apply_client() {
+	if [[ $ARCH == "arm64" ]]; then
+	    yq eval '.spec.template.spec.tolerations += '"$TOLERATIONS_ARM64"'' "$1" | kubectl_bin apply -f -


[shfmt] _{reported by reviewdog 🐶}

Suggested change

yq eval '.spec.template.spec.tolerations += '"$TOLERATIONS_ARM64"'' "$1" | kubectl_bin apply -f -

yq eval '.spec.template.spec.tolerations += '"$TOLERATIONS_ARM64"'' "$1" | kubectl_bin apply -f -

JNKPercona · 2024-12-20T13:34:51Z

Test name	Status
arbiter	passed
balancer	passed
custom-replset-name	passed
custom-tls	passed
custom-users-roles	passed
custom-users-roles-sharded	passed
cross-site-sharded	passed
data-at-rest-encryption	failure
data-sharded	passed
demand-backup	passed
demand-backup-fs	passed
demand-backup-eks-credentials-irsa	passed
demand-backup-physical	passed
demand-backup-physical-sharded	passed
demand-backup-sharded	passed
expose-sharded	passed
ignore-labels-annotations	passed
init-deploy	passed
finalizer	passed
ldap	passed
ldap-tls	passed
limits	passed
liveness	passed
mongod-major-upgrade	passed
mongod-major-upgrade-sharded	passed
monitoring-2-0	passed
multi-cluster-service	passed
non-voting	passed
one-pod	passed
operator-self-healing-chaos	passed
pitr	passed
pitr-sharded	passed
pitr-physical	passed
pvc-resize	passed
recover-no-primary	passed
replset-overrides	passed
rs-shard-migration	passed
scaling	passed
scheduled-backup	passed
security-context	passed
self-healing-chaos	passed
service-per-pod	passed
serviceless-external-nodes	passed
smart-update	passed
split-horizon	passed
stable-resource-version	passed
storage	passed
tls-issue-cert-manager	passed
upgrade	passed
upgrade-consistency	passed
upgrade-consistency-sharded-tls	passed
upgrade-sharded	passed
users	passed
version-service	passed
We run 54 out of 54

commit: 268b731
image: perconalab/percona-server-mongodb-operator:PR-1735-268b7313

eleo007 and others added 13 commits November 11, 2024 12:08

Merge pull request #1714 from percona/release-1.18.0-push-test

97a6f79

Fix tests for 1.18.0 release

support for running e2e tests against arm64 GKE nodes

dc3633e

making cert-manager able to be installed on arm64 nodes (using helm)

0a03ce9

use apply_clien always so that client pod has proper tolerations applied

1111fdc

update e2e test to use alpine/curl with tolerations for arm64 archite…

be2c942

…cture

use apply_clien always so that client pod has proper tolerations appl…

970942a

…ied - 02

use apply_clien always so that client pod has proper tolerations appl…

da6966d

…ied - 03

use apply_clien always so that client pod has proper tolerations appl…

09d5151

…ied - 04

update deploy_cmctl function to handle arm64 architecture tolerations

d49669e

use apply_clien always so that client pod has proper tolerations appl…

f56e773

…ied - 05

add tolerations for arm64 architecture in run_simple_cli_inside_image…

b1a00f2

… function

add tolerations for arm64 architecture in deploy_operator_gh and clea…

908eb0f

…n up apply_cluster logic

add initial run-release-arm64 configuration with various options

4c0c72d

pull-request-size bot added the size/XL 500-999 lines label Nov 20, 2024

Merge branch 'release-1.18.0' into K8SPSMDB-1213

e7eeeab

github-actions bot reviewed Nov 20, 2024

View reviewed changes

e2e-tests/upgrade-sharded/run Show resolved Hide resolved

e2e-tests/upgrade/run Show resolved Hide resolved

ptankov changed the title ~~K8 spsmdb 1213~~ K8SPSMDB-1213 Add arm64 support for the e2e tests Nov 20, 2024

amazon/aws-cli instead of perconalab/awscli

43c416b

github-actions bot reviewed Nov 21, 2024

View reviewed changes

e2e-tests/storage/run Show resolved Hide resolved

add support for arm64 architecture in various e2e tests and enhance b…

f9282dd

…ackup checks

pull-request-size bot added size/XXL 1000+ lines and removed size/XL 500-999 lines labels Nov 25, 2024

github-actions bot reviewed Nov 25, 2024

View reviewed changes

e2e-tests/default-cr/run Show resolved Hide resolved

e2e-tests/default-cr/run Show resolved Hide resolved

e2e-tests/default-cr/run Show resolved Hide resolved

e2e-tests/default-cr/run Show resolved Hide resolved

e2e-tests/default-cr/run Show resolved Hide resolved

for upgrade-consistency and upgrade-consistency-sharded-tls tests, we…

ee95241

… also need to rename compare files in addition to update of run file content

ptankov marked this pull request as ready for review November 25, 2024 10:22

ptankov requested review from nmarukovich, jvpasinatto, eleo007, hors and egegunes as code owners November 25, 2024 10:22

ptankov added 4 commits November 27, 2024 17:45

Refactor cert-manager deployment to use kubectl patch for arm64 toler…

fe10601

…ations

Update deploy_cmctl function to use service account for RBAC configur…

35b3167

…ation

Remove arm64 toleration from chaos pod failure configuration in self-…

48ab905

…healing tests

Set cluster variable for telemetry transfer function in e2e tests

b25251c

ptankov force-pushed the K8SPSMDB-1213 branch from 74b47dd to b25251c Compare November 29, 2024 12:37

eleo007 reviewed Nov 29, 2024

View reviewed changes

cw-rbac.yaml.1 Outdated Show resolved Hide resolved

cw-rbac.yaml Outdated Show resolved Hide resolved

cw-rbac.yaml.2 Outdated Show resolved Hide resolved

e2e-tests/data-sharded/run Outdated Show resolved Hide resolved

e2e-tests/default-cr/run Outdated Show resolved Hide resolved

Committed by mistake. Removed.

bf9d74c

pull-request-size bot added size/XL 500-999 lines and removed size/XXL 1000+ lines labels Dec 11, 2024

ptankov added 5 commits December 11, 2024 10:39

Merge branch 'main' into K8SPSMDB-1213

74ba5cb

returning commented out code, used for debugging purposes

f78625c

Merge branch 'main' into K8SPSMDB-1213

4666fc9

refactor: replace hardcoded tolerations with variable for arm64 archi…

dc76f30

…tecture

Merge branch 'K8SPSMDB-1213' of github.com:percona/percona-server-mon…

6bc5de5

…godb-operator into K8SPSMDB-1213

github-actions bot reviewed Dec 11, 2024

View reviewed changes

e2e-tests/default-cr/run Show resolved Hide resolved

e2e-tests/default-cr/run Show resolved Hide resolved

e2e-tests/functions Show resolved Hide resolved

ptankov added 2 commits December 11, 2024 12:01

refactor: simplify architecture check by removing quotes around varia…

592bbdc

…ble to satisfy reviewdog

refactor: use regex for architecture check to improve readability and…

15d6819

… satisfy reviewdog

ptankov requested a review from eleo007 December 11, 2024 10:03

github-actions bot reviewed Dec 11, 2024

View reviewed changes

e2e-tests/default-cr/run Show resolved Hide resolved

e2e-tests/default-cr/run Show resolved Hide resolved

e2e-tests/ldap-tls/run Show resolved Hide resolved

Merge branch 'main' into K8SPSMDB-1213

175f6d6

github-actions bot reviewed Dec 11, 2024

View reviewed changes

e2e-tests/balancer/run Show resolved Hide resolved

e2e-tests/functions Show resolved Hide resolved

e2e-tests/functions Show resolved Hide resolved

e2e-tests/functions Show resolved Hide resolved

Merge branch 'main' into K8SPSMDB-1213

352cebd

eleo007 reviewed Dec 19, 2024

View reviewed changes

e2e-tests/serviceless-external-nodes/run Show resolved Hide resolved

ptankov added 4 commits December 19, 2024 14:04

Merge branch 'main' into K8SPSMDB-1213

13eb721

resolving the bug about applying secrets after creating cluste

048db89

Merge branch 'main' into K8SPSMDB-1213

55ff954

Merge branch 'main' into K8SPSMDB-1213

268b731

github-actions bot reviewed Dec 20, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

K8SPSMDB-1213 Add arm64 support for the e2e tests #1735

K8SPSMDB-1213 Add arm64 support for the e2e tests #1735

ptankov commented Nov 20, 2024 •

edited

Loading

it-percona-cla commented Dec 4, 2024

github-actions bot Dec 20, 2024

github-actions bot Dec 20, 2024

github-actions bot Dec 20, 2024

github-actions bot Dec 20, 2024

JNKPercona commented Dec 20, 2024

	.spec.upgradeOptions.apply="Never"' "$1" > $temp_cr
	.spec.upgradeOptions.apply="Never"' "$1" >$temp_cr

	yq eval '.spec.replsets[].tolerations += '"$TOLERATIONS_ARM64"' \|
	yq eval '.spec.replsets[].tolerations += '"$TOLERATIONS_ARM64"' \|

	yq eval '.spec.template.spec.tolerations += '"$TOLERATIONS_ARM64"'' "$1" \| kubectl_bin apply -f -
	yq eval '.spec.template.spec.tolerations += '"$TOLERATIONS_ARM64"'' "$1" \| kubectl_bin apply -f -

K8SPSMDB-1213 Add arm64 support for the e2e tests #1735

Are you sure you want to change the base?

K8SPSMDB-1213 Add arm64 support for the e2e tests #1735

Conversation

ptankov commented Nov 20, 2024 • edited Loading

CHANGE DESCRIPTION

CHECKLIST

it-percona-cla commented Dec 4, 2024

github-actions bot Dec 20, 2024

Choose a reason for hiding this comment

github-actions bot Dec 20, 2024

Choose a reason for hiding this comment

github-actions bot Dec 20, 2024

Choose a reason for hiding this comment

github-actions bot Dec 20, 2024

Choose a reason for hiding this comment

JNKPercona commented Dec 20, 2024

ptankov commented Nov 20, 2024 •

edited

Loading