Adding s390x arch support in k8s 1.28 provider #1201

chandramerla · 2024-06-11T06:27:42Z

What this PR does / why we need it:
This PR enables kubernetes 1.28 provider to be s390x arch compatible in SLIM(=true) mode. This is one of the initial steps to enabling Kubevirt CI for s390x

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:
With these changes I've run all the steps in KUBEVIRTCI_LOCAL_TESTING.md guide to test provider in both x86 and s390x machines, with specific mention of some details as below:

Have not tested with KSM and SWAP ON and keeping SLIM=true for s390x, as this initial PR is targeting to add support of s390x for 1.28 provider in slim mode.
Tested make cluster-up on s390x with even in single stack mode
Have tested make cluster-sync on x86 (with KUBEVIRT_CDI_DEPLOY=false as otherwise there is an error 'error when creating "..../uploadproxy-nodeport.yaml": namespaces "cdi" not found')
make functests on x86 are failing locally with 'timer expired' errors, etc., after I resolve the issue mentioned following the workaround

Checklist

This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

Design: A design document was considered and is present (link) or not required
PR: The PR description is expressive enough and will help future contributors
Code: Write code that humans can understand and Keep it simple
Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
Upgrade: Impact of this change on upgrade flows was considered and addressed if required
Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
For s390x also one can follow KUBEVIRTCI_LOCAL_TESTING.md as he/she does for x86. So, no additional doc is needed.
Community: Announcement to kubevirt-dev was considered

Release note:

NONE

* Used qemu-monitor command to add secondary network interface to avoid the primary network interface having names other than eth0. As based on the MAC address of eth0 interface we are fetching the IP of the VM from DHCP server, it needs to be MAC of the primary network interface, to obtain the IP for the interface. * Bumped calico CNI version from 3.18.0 to 3.27.2 (latest currently), as 3.18.0 doesn't support s390x arch. Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

kubevirt-bot · 2024-06-11T06:27:50Z

Hi @chandramerla. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

kubevirt-bot · 2024-06-11T06:27:51Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign qinqon for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

chandramerla · 2024-06-11T06:35:28Z

/test ?

kubevirt-bot · 2024-06-11T06:35:43Z

@chandramerla: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

chandramerla · 2024-06-11T06:37:24Z

/cc @brianmcarey

chandramerla · 2024-06-11T06:40:36Z

/unassign @brianmcarey

chandramerla · 2024-06-11T06:41:01Z

/cc @brianmcarey

chandramerla · 2024-06-11T06:43:40Z

@ Reviewers: Happy to take you through the changes in a call if that helps. Please let me know. Thanks!

brianmcarey · 2024-06-18T06:59:25Z

/test check-provision-k8s-1.28

Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

This reverts commit 3cdfc8f. Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

brianmcarey · 2024-06-26T09:14:02Z

/test check-provision-k8s-1.28

brianmcarey · 2024-06-26T09:14:32Z

/test ?

kubevirt-bot · 2024-06-26T09:14:35Z

@brianmcarey: The following commands are available to trigger required jobs:

/test check-provision-alpine-with-test-tooling
/test check-provision-centos-base
/test check-provision-k8s-1.28
/test check-provision-k8s-1.29
/test check-provision-k8s-1.30
/test check-provision-manager
/test check-up-kind-ovn
/test check-up-kind-sriov

The following commands are available to trigger optional jobs:

/test check-gocli
/test check-up-kind-1.27-vgpu

Use /test all to run the following jobs that were automatically triggered:

check-provision-alpine-with-test-tooling
check-provision-k8s-1.28
check-provision-k8s-1.29
check-provision-k8s-1.30
check-provision-manager
check-up-kind-1.27-vgpu
check-up-kind-sriov

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

brianmcarey · 2024-06-26T09:15:11Z

/test check-gocli
/test check-provision-centos-base

…90x are broken due to various issues Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

…arg to docker build Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

brianmcarey

Thanks for this @chandramerla

My main worry with this is that the k8s-1.28 provider is soon to be removed as we have started adding the k8s-1.31.

It may be better to target the k8s-1.30 provider.

brianmcarey · 2024-07-03T09:33:27Z

KUBEVIRTCI_LOCAL_TESTING.md

@@ -48,7 +48,7 @@ export KUBEVIRTCI_GOCLI_CONTAINER=quay.io/kubevirtci/gocli:latest
 ### start cluster

 ```bash
-export KUBEVIRT_PROVIDER=k8s-1.30
+export KUBEVIRT_PROVIDER=k8s-1.28


Why are you dropping the version here?

@brianmcarey I just made the version consistent across this file :)
I saw in a place it is 1.27

(cd cluster-provision/k8s/1.28; ../provision.sh)
In other two places as 1.30
export KUBEVIRT_PROVIDER=k8s-1.30
export KUBEVIRT_PROVIDER=k8s-1.30
In other two places 1.21
export PHASES=k8s; (cd cluster-provision/k8s/1.21; ../provision.sh)

So everywhere I changed to 1.28

Let me change all these to 1.30

addressed in k8s-1.30-provider-slim-s390x branch

brianmcarey · 2024-07-03T09:34:50Z

KUBEVIRTCI_LOCAL_TESTING.md

 Run the `k8s` step as much as needed. It reuses the intermediate image that was created
 by the `linux` phase.  
+Note :
+1. By default when you run `k8s` phase alone, it uses centos9 image specified in cluster-provision/k8s/base-image, not the one built locally in the `linux` phase. So, to make `k8s` phase use the locally built centos9 image, update cluster-provision/k8s/base-image with the locally built image name and tag (default: quay.io/kubevirtci/centos9:latest)


thanks for adding this

brianmcarey · 2024-07-03T09:38:01Z

cluster-provision/centos9/Dockerfile

+      curl -L -o dockerize-linux-$BUILDARCH.tar.gz https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz; \
+    else \
+      # Temporary till s390x support is upstreamed to dockerize (https://github.com/jwilder/dockerize/pull/200)
+      curl -L -o dockerize-linux-$BUILDARCH.tar.gz https://github.com/ibm-jitendra/kubevirt_pkgs/raw/main/dockerize-linux-s390x.tar.gz; \


Is there any timeline for the upstreaming of this? I would prefer not to rely on a random tarball.

@brianmcarey : No update as of now. We are tried to reach upstream owner on PR, but no luck. Now, Vamsi is trying to reach him on LinkedIn.

Thanks for @vamsikrishna-siddu for the followup. Now changes are upstreamed.
This is addressed in k8s-1.30-provider-slim-s390x branch

brianmcarey · 2024-07-03T09:40:20Z

cluster-provision/centos9/scripts/kernel.s390x.args

@@ -0,0 +1 @@
+root=/dev/vda1 ro no_timer_check console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M


It would be good to have a separate commit for these kernel args that explains why some of the different options are required.

These were the defaults within the generic cloud image of centos9 for s390x. I've copied them to this file to mainly keep parity with x86 based kernel args, which I think exists for configuring args for kernel.

Will try to come up with a readme around this.

Added this as separate commit in k8s-1.30-provider-slim-s390x branch, where in commit I've explained all the details.
6f3f133

brianmcarey · 2024-07-03T09:43:36Z

cluster-provision/centos9/scripts/vm.sh

+    -device AC97 \
+    -uuid $(cat /proc/sys/kernel/random/uuid) \
+    ${QEMU_ARGS}"
+else


I would prefer that the else would cover the more common x86_64 case

If I understand you right, have s390x case in if block and x86_64 case in else block. I will change it accordingly.

This is addressed in k8s-1.30-provider-slim-s390x branch

brianmcarey · 2024-07-29T08:21:21Z

cluster-provision/gocli/cmd/provision.go

@@ -252,21 +255,21 @@ func provisionCluster(cmd *cobra.Command, args []string) (retErr error) {
 		if err != nil {
 			return err
 		}
-		err = _cmd(cli, nodeContainer(prefix, nodeName), "if [ -f /scripts/extra-pre-pull-images ]; then scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i vagrant.key -P 22 /scripts/extra-pre-pull-images vagrant@192.168.66.101:/tmp/extra-pre-pull-images; fi", "copying /scripts/extra-pre-pull-images if existing")
+		err = _cmd(cli, nodeContainer(prefix, nodeName), fmt.Sprintf("if [ -f /scripts/extra-pre-pull-images ]; then scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i vagrant.key -P 22 /scripts/extra-pre-pull-images %s@192.168.66.101:/tmp/extra-pre-pull-images; fi", sshUser), "copying /scripts/extra-pre-pull-images if existing")


I think some of this ssh handling has changed since #1209 was merged

@brianmcarey This PR changes are already part of my branch. But will check if I need to adjust it. As PR#1209 is quite big digesting it still.

We don't require to change this as in #1029 these function calls have not changed.

brianmcarey · 2024-07-29T08:27:53Z

cluster-provision/k8s/1.28/k8s_provision.sh

@@ -301,6 +315,11 @@ kubeadm_raw_ipv6=/tmp/kubeadm_ipv6.conf
 kubeadm_manifest="/etc/kubernetes/kubeadm.conf"
 kubeadm_manifest_ipv6="/etc/kubernetes/kubeadm_ipv6.conf"

+# envsubst pkg is not available by default in s390x Architecture.
+if [ "$arch" == "s390x" ]; then 
+   dnf install -y gettext


We could probably include this when we install this with other rpm dependencies in provision.sh

I don't think we need an if statement here - if its already installed on x86 it won't do anything anyway.

Addressed in k8s-1.30-provider-slim-s390x branch, by moving the line dnf install -y gettext to the end of provision.sh

brianmcarey · 2024-07-29T08:29:07Z

cluster-provision/k8s/1.28/provision.sh

@@ -14,13 +16,13 @@ export ISTIO_BIN_DIR="/opt/istio-${ISTIO_VERSION}/bin"
 EOF
 source $KUBEVIRTCI_SHARED_DIR/shared_vars.sh

-# Install modules of the initrd kernel
+# Install modules of the initrd kernel. These modules extend the kernel's functionality, providing support for various hardware devices, file systems, network protocols, KVM/virtualization, and other features.


I'm not sure if we need this additional comment

Will remove it. Added it as initially for my reference to understand the sequence of steps

Addressed in k8s-1.30-provider-slim-s390x branch

brianmcarey · 2024-07-29T08:29:23Z

cluster-provision/k8s/1.28/provision.sh

 dnf install -y "kernel-modules-$(uname -r)"

 # Resize root partition
 dnf install -y cloud-utils-growpart
-if growpart /dev/vda 1; then
-    resize2fs /dev/vda1
+if growpart /dev/vda 1; then #growpart adjusts the partition size to fill the available space on the disk


same here - not sure if these additional comments are needed

Will remove it

Addressed in k8s-1.30-provider-slim-s390x branch

brianmcarey · 2024-07-29T08:30:12Z

cluster-provision/k8s/1.28/provision.sh

-dnf install -y openvswitch2.16
+#openvswitch for s390x is not available from the centos default repos.
+if [ "$ARCH" == "s390x" ]; then
+  dnf install -y https://kojipkgs.fedoraproject.org//packages/openvswitch/2.16.0/2.fc36/s390x/openvswitch-2.16.0-2.fc36.s390x.rpm


is this the best available source for openvswitch for s390x?

Yes, unfortunately.
I checked today also, couldn't find it else where.

chandramerla

@brianmcarey Thanks for your review comments. I've added my initial replies. I will work on them as in my replies.

Also as we discussed earlier, I've added all changes required for publishing multi-arch gocli, centos9 and k8s provider images at here and at kubevirt/project-infra#3566. I've kept multi-arch publish related changes required in kubevirtci repo separately here, so that it is easier for review. But if you prefer merging it to this #1201 PR for review, please let me know. I'll be awaiting for your feedback on this.

On targeting 1.30 provider for enable for s390x, I will create a separate branch and start doing changes there. The issue that stopped me earlier to start with newer providers for s390x support for k8s equivalent crio version builds aren't available or broken for s390x. This time if I face such issues, I'm thinking to try with containerd in place of crio just to unblock.

chandramerla · 2024-07-31T06:02:30Z

KUBEVIRTCI_LOCAL_TESTING.md

@@ -48,7 +48,7 @@ export KUBEVIRTCI_GOCLI_CONTAINER=quay.io/kubevirtci/gocli:latest
 ### start cluster

 ```bash
-export KUBEVIRT_PROVIDER=k8s-1.30
+export KUBEVIRT_PROVIDER=k8s-1.28


@brianmcarey I just made the version consistent across this file :)
I saw in a place it is 1.27

(cd cluster-provision/k8s/1.28; ../provision.sh)
In other two places as 1.30
export KUBEVIRT_PROVIDER=k8s-1.30
export KUBEVIRT_PROVIDER=k8s-1.30
In other two places 1.21
export PHASES=k8s; (cd cluster-provision/k8s/1.21; ../provision.sh)

So everywhere I changed to 1.28

Let me change all these to 1.30

chandramerla · 2024-07-31T06:02:44Z

KUBEVIRTCI_LOCAL_TESTING.md

@@ -59,7 +59,7 @@ make cluster-up
 #### start cluster with prometheus, alertmanager and grafana
 To enable prometheus, please also export the following variables before running `make cluster-up`:
 ```bash
-export KUBEVIRT_PROVIDER=k8s-1.30
+export KUBEVIRT_PROVIDER=k8s-1.28


Yes, will change

chandramerla · 2024-07-31T06:07:15Z

cluster-provision/centos9/Dockerfile

+      curl -L -o /initramfs-amd64.img http://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/images/pxeboot/initrd.img && \
+      curl -L -o /vmlinuz-amd64 http://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/images/pxeboot/vmlinuz; \
+    else \
+      /download_box.sh https://cloud.centos.org/centos/9-stream/s390x/images/CentOS-Stream-GenericCloud-9-$centos_version.s390x.qcow2 && \


@brianmcarey I think we can change, but just want to not to mix those changes for x86 here. As this is already XXL, wanted to have that as a separate PR. I will create that as a follow-up PR for this one.

chandramerla · 2024-07-31T06:28:17Z

cluster-provision/centos9/Dockerfile

+      curl -L -o dockerize-linux-$BUILDARCH.tar.gz https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz; \
+    else \
+      # Temporary till s390x support is upstreamed to dockerize (https://github.com/jwilder/dockerize/pull/200)
+      curl -L -o dockerize-linux-$BUILDARCH.tar.gz https://github.com/ibm-jitendra/kubevirt_pkgs/raw/main/dockerize-linux-s390x.tar.gz; \


@brianmcarey : No update as of now. We are tried to reach upstream owner on PR, but no luck. Now, Vamsi is trying to reach him on LinkedIn.

chandramerla · 2024-07-31T06:34:37Z

cluster-provision/centos9/scripts/kernel.s390x.args

@@ -0,0 +1 @@
+root=/dev/vda1 ro no_timer_check console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M


These were the defaults within the generic cloud image of centos9 for s390x. I've copied them to this file to mainly keep parity with x86 based kernel args, which I think exists for configuring args for kernel.

Will try to come up with a readme around this.

chandramerla · 2024-07-31T06:40:25Z

cluster-provision/centos9/scripts/vm.sh

+    -device AC97 \
+    -uuid $(cat /proc/sys/kernel/random/uuid) \
+    ${QEMU_ARGS}"
+else


If I understand you right, have s390x case in if block and x86_64 case in else block. I will change it accordingly.

chandramerla · 2024-08-01T15:20:22Z

cluster-provision/gocli/cmd/provision.go

@@ -252,21 +255,21 @@ func provisionCluster(cmd *cobra.Command, args []string) (retErr error) {
 		if err != nil {
 			return err
 		}
-		err = _cmd(cli, nodeContainer(prefix, nodeName), "if [ -f /scripts/extra-pre-pull-images ]; then scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i vagrant.key -P 22 /scripts/extra-pre-pull-images vagrant@192.168.66.101:/tmp/extra-pre-pull-images; fi", "copying /scripts/extra-pre-pull-images if existing")
+		err = _cmd(cli, nodeContainer(prefix, nodeName), fmt.Sprintf("if [ -f /scripts/extra-pre-pull-images ]; then scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i vagrant.key -P 22 /scripts/extra-pre-pull-images %s@192.168.66.101:/tmp/extra-pre-pull-images; fi", sshUser), "copying /scripts/extra-pre-pull-images if existing")


@brianmcarey This PR changes are already part of my branch. But will check if I need to adjust it. As PR#1209 is quite big digesting it still.

chandramerla · 2024-08-01T15:23:06Z

cluster-provision/k8s/1.28/provision.sh

@@ -14,13 +16,13 @@ export ISTIO_BIN_DIR="/opt/istio-${ISTIO_VERSION}/bin"
 EOF
 source $KUBEVIRTCI_SHARED_DIR/shared_vars.sh

-# Install modules of the initrd kernel
+# Install modules of the initrd kernel. These modules extend the kernel's functionality, providing support for various hardware devices, file systems, network protocols, KVM/virtualization, and other features.


Will remove it. Added it as initially for my reference to understand the sequence of steps

chandramerla · 2024-08-01T15:23:08Z

cluster-provision/k8s/1.28/provision.sh

 dnf install -y "kernel-modules-$(uname -r)"

 # Resize root partition
 dnf install -y cloud-utils-growpart
-if growpart /dev/vda 1; then
-    resize2fs /dev/vda1
+if growpart /dev/vda 1; then #growpart adjusts the partition size to fill the available space on the disk


Will remove it

chandramerla · 2024-08-01T15:53:10Z

cluster-provision/k8s/1.28/provision.sh

-dnf install -y openvswitch2.16
+#openvswitch for s390x is not available from the centos default repos.
+if [ "$ARCH" == "s390x" ]; then
+  dnf install -y https://kojipkgs.fedoraproject.org//packages/openvswitch/2.16.0/2.fc36/s390x/openvswitch-2.16.0-2.fc36.s390x.rpm


Yes, unfortunately.
I checked today also, couldn't find it else where.

Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

kubevirt-bot · 2024-08-11T19:38:18Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

chandramerla

@brianmcarey
I've addressed all your comments. As 1.28 is going away soon, created another branch where I've moved all these changes (+ addressing your comments) for k8s 1.30 provider. The PR for the same is : #1252

chandramerla · 2024-08-08T04:02:21Z

KUBEVIRTCI_LOCAL_TESTING.md

@@ -48,7 +48,7 @@ export KUBEVIRTCI_GOCLI_CONTAINER=quay.io/kubevirtci/gocli:latest
 ### start cluster

 ```bash
-export KUBEVIRT_PROVIDER=k8s-1.30
+export KUBEVIRT_PROVIDER=k8s-1.28


addressed in k8s-1.30-provider-slim-s390x branch

chandramerla · 2024-08-08T04:04:06Z

KUBEVIRTCI_LOCAL_TESTING.md

@@ -59,7 +59,7 @@ make cluster-up
 #### start cluster with prometheus, alertmanager and grafana
 To enable prometheus, please also export the following variables before running `make cluster-up`:
 ```bash
-export KUBEVIRT_PROVIDER=k8s-1.30
+export KUBEVIRT_PROVIDER=k8s-1.28


addressed in k8s-1.30-provider-slim-s390x branch

chandramerla · 2024-08-08T04:31:15Z

cluster-provision/centos9/Dockerfile

+      curl -L -o /initramfs-amd64.img http://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/images/pxeboot/initrd.img && \
+      curl -L -o /vmlinuz-amd64 http://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/images/pxeboot/vmlinuz; \
+    else \
+      /download_box.sh https://cloud.centos.org/centos/9-stream/s390x/images/CentOS-Stream-GenericCloud-9-$centos_version.s390x.qcow2 && \


Added the placeholder PR #1242 so that this can be further tracked there.

chandramerla · 2024-08-08T04:40:08Z

cluster-provision/centos9/Dockerfile

+      curl -L -o dockerize-linux-$BUILDARCH.tar.gz https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz; \
+    else \
+      # Temporary till s390x support is upstreamed to dockerize (https://github.com/jwilder/dockerize/pull/200)
+      curl -L -o dockerize-linux-$BUILDARCH.tar.gz https://github.com/ibm-jitendra/kubevirt_pkgs/raw/main/dockerize-linux-s390x.tar.gz; \


Thanks for @vamsikrishna-siddu for the followup. Now changes are upstreamed.
This is addressed in k8s-1.30-provider-slim-s390x branch

chandramerla · 2024-08-08T06:19:01Z

cluster-provision/centos9/scripts/kernel.s390x.args

@@ -0,0 +1 @@
+root=/dev/vda1 ro no_timer_check console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M


Added this as separate commit in k8s-1.30-provider-slim-s390x branch, where in commit I've explained all the details.
6f3f133

chandramerla · 2024-08-08T10:29:18Z

cluster-provision/centos9/scripts/vm.sh

+    -device AC97 \
+    -uuid $(cat /proc/sys/kernel/random/uuid) \
+    ${QEMU_ARGS}"
+else


This is addressed in k8s-1.30-provider-slim-s390x branch

chandramerla · 2024-08-08T22:51:57Z

cluster-provision/gocli/cmd/provision.go

@@ -252,21 +255,21 @@ func provisionCluster(cmd *cobra.Command, args []string) (retErr error) {
 		if err != nil {
 			return err
 		}
-		err = _cmd(cli, nodeContainer(prefix, nodeName), "if [ -f /scripts/extra-pre-pull-images ]; then scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i vagrant.key -P 22 /scripts/extra-pre-pull-images vagrant@192.168.66.101:/tmp/extra-pre-pull-images; fi", "copying /scripts/extra-pre-pull-images if existing")
+		err = _cmd(cli, nodeContainer(prefix, nodeName), fmt.Sprintf("if [ -f /scripts/extra-pre-pull-images ]; then scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i vagrant.key -P 22 /scripts/extra-pre-pull-images %s@192.168.66.101:/tmp/extra-pre-pull-images; fi", sshUser), "copying /scripts/extra-pre-pull-images if existing")


We don't require to change this as in #1029 these function calls have not changed.

chandramerla · 2024-08-09T03:59:48Z

cluster-provision/k8s/1.28/provision.sh

@@ -14,13 +16,13 @@ export ISTIO_BIN_DIR="/opt/istio-${ISTIO_VERSION}/bin"
 EOF
 source $KUBEVIRTCI_SHARED_DIR/shared_vars.sh

-# Install modules of the initrd kernel
+# Install modules of the initrd kernel. These modules extend the kernel's functionality, providing support for various hardware devices, file systems, network protocols, KVM/virtualization, and other features.


Addressed in k8s-1.30-provider-slim-s390x branch

chandramerla · 2024-08-09T04:02:14Z

cluster-provision/k8s/1.28/provision.sh

 dnf install -y "kernel-modules-$(uname -r)"

 # Resize root partition
 dnf install -y cloud-utils-growpart
-if growpart /dev/vda 1; then
-    resize2fs /dev/vda1
+if growpart /dev/vda 1; then #growpart adjusts the partition size to fill the available space on the disk


Addressed in k8s-1.30-provider-slim-s390x branch

chandramerla · 2024-08-09T05:33:57Z

cluster-provision/k8s/1.28/k8s_provision.sh

@@ -301,6 +315,11 @@ kubeadm_raw_ipv6=/tmp/kubeadm_ipv6.conf
 kubeadm_manifest="/etc/kubernetes/kubeadm.conf"
 kubeadm_manifest_ipv6="/etc/kubernetes/kubeadm_ipv6.conf"

+# envsubst pkg is not available by default in s390x Architecture.
+if [ "$arch" == "s390x" ]; then 
+   dnf install -y gettext


Addressed in k8s-1.30-provider-slim-s390x branch, by moving the line dnf install -y gettext to the end of provision.sh

chandramerla · 2024-09-16T09:16:26Z

Closing this PR as 1.28 is no more part of k8s providers and also not these changes are moved to 1.30 provider in the PR: #1252

kubevirt-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Jun 11, 2024

kubevirt-bot requested review from davidvossel and dhiller June 11, 2024 06:27

kubevirt-bot added the size/XXL label Jun 11, 2024

kubevirt-bot assigned brianmcarey Jun 11, 2024

kubevirt-bot unassigned brianmcarey Jun 11, 2024

kubevirt-bot requested a review from brianmcarey June 11, 2024 06:41

kubevirt-bot added dco-signoff: no Indicates the PR's author has not DCO signed all their commits. and removed dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Jun 25, 2024

Correct if condition syntax in cluster-up/check.sh

3cdfc8f

Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

chandramerla force-pushed the k8s-1.28-provider-minimal branch from 68dde11 to 3cdfc8f Compare June 25, 2024 12:14

kubevirt-bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. and removed dco-signoff: no Indicates the PR's author has not DCO signed all their commits. labels Jun 25, 2024

Revert "Correct if condition syntax in cluster-up/check.sh"

0f3e334

This reverts commit 3cdfc8f. Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 16, 2024

chandramerla added 3 commits July 18, 2024 11:22

Updated the crio version for s390x to 1.28.4 as other versions for s3…

1873650

…90x are broken due to various issues Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

Corrected if syntax in check.sh

6c6a658

Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

Synced from kubevirt/kubevirtci/main

f63f97e

Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 18, 2024

chandramerla marked this pull request as ready for review July 19, 2024 04:54

kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 19, 2024

kubevirt-bot requested review from rmohr and vladikr July 19, 2024 04:54

Corrected some warnings in Dockerfile and passing BUILDARCH as build-…

406fdaf

…arg to docker build Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

brianmcarey reviewed Jul 29, 2024

View reviewed changes

chandramerla commented Aug 1, 2024

View reviewed changes

Use username for NewSSHClient based on architecture

8d33a40

Signed-off-by: chandramerla <Chandra.Merla@ibm.com>

kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 11, 2024

chandramerla mentioned this pull request Aug 21, 2024

K8s 1.30 provider slim s390x #1252

Merged

8 tasks

chandramerla commented Aug 21, 2024

View reviewed changes

chandramerla closed this Sep 16, 2024

		@@ -0,0 +1 @@
		root=/dev/vda1 ro no_timer_check console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M

Adding s390x arch support in k8s 1.28 provider #1201

Adding s390x arch support in k8s 1.28 provider #1201

Conversation

chandramerla commented Jun 11, 2024

kubevirt-bot commented Jun 11, 2024

kubevirt-bot commented Jun 11, 2024

chandramerla commented Jun 11, 2024

kubevirt-bot commented Jun 11, 2024

chandramerla commented Jun 11, 2024 • edited Loading

chandramerla commented Jun 11, 2024

chandramerla commented Jun 11, 2024

chandramerla commented Jun 11, 2024 • edited Loading

brianmcarey commented Jun 18, 2024

brianmcarey commented Jun 26, 2024

brianmcarey commented Jun 26, 2024

kubevirt-bot commented Jun 26, 2024

brianmcarey commented Jun 26, 2024

brianmcarey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chandramerla left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kubevirt-bot commented Aug 11, 2024

chandramerla left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chandramerla commented Sep 16, 2024

chandramerla commented Jun 11, 2024 •

edited

Loading

chandramerla commented Jun 11, 2024 •

edited

Loading

chandramerla left a comment •

edited

Loading

chandramerla left a comment •

edited

Loading