[Release-1.28] - Kine TLS support for k3s

[vitorsavian]

Backport fix for Kine TLS support for k3s

• Kine TLS support for k3s #9563

[VestigeJ]

##Environment Details
Reproduced using VERSION=v1.28.8+k3s1
Validated using COMMIT=70bd37793f0eadb9de3f8e5a224d2a9eb75c38b7

Infrastructure

• Cloud

Node(s) CPU architecture, OS, and version:

Linux 5.14.21-150500.53-default x86_64 GNU/Linux
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"

Cluster Configuration:

NAME STATUS ROLES AGE VERSION
ip-1-3-7-1 Ready control-plane,master 114s v1.28.8+k3s-70bd3779

Config.yaml:

node-external-ip: 1.3.7.1
token: YOUR_TOKEN_HERE
write-kubeconfig-mode: 644
debug: true
protect-kernel-defaults: true
embedded-registry: true
kine-tls: true

Reproduction

$ curl https://get.k3s.io --output install-"k3s".sh
$ sudo chmod +x install-"k3s".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/90-kubelet.conf
$ sudo cp 90-kubelet.conf /etc/sysctl.d/
$ sudo systemctl restart systemd-sysctl
$ COMMIT=70bd37793f0eadb9de3f8e5a224d2a9eb75c38b7
$ sudo INSTALL_K3S_COMMIT=$COMMIT INSTALL_K3S_EXEC="server --kine-tls" ./install-k3s.sh
$ set_kubefig
$ kgp -A -o wide
$ sudo etcdctl endpoint status --cacert=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --endpoints=unixs:///var/lib/rancher/k3s/server/kine.sock -w table

Results:

Existing release does not support kine-tls - if it's present in the config.yaml it will be ignored and start up correctly without the feature fwiw.

$ sudo INSTALL_K3S_VERSION=$VERSION INSTALL_K3S_EXEC="server --kine-tls" ./install-k3s.sh

[INFO]  Using v1.28.8+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.28.8+k3s1/sha256sum-amd64.txt
[INFO]  Skipping binary downloaded, installed k3s matches hash
[INFO]  Skipping installation of SELinux RPM
[INFO]  Skipping /usr/local/bin/kubectl symlink to k3s, already exists
[INFO]  Skipping /usr/local/bin/crictl symlink to k3s, already exists
[INFO]  Skipping /usr/local/bin/ctr symlink to k3s, already exists
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s
Job for k3s.service failed because the control process exited with error code.
See "systemctl status k3s.service" and "journalctl -xeu k3s.service" for details.

// new behavior correctly starts the kine endpoint locally.

$ sudo etcdctl endpoint status --cacert=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --endpoints=unixs:///var/lib/rancher/k3s/server/kine.sock -w table

+-----------------------------------------------+----+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|                   ENDPOINT                    | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+-----------------------------------------------+----+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| unixs:///var/lib/rancher/k3s/server/kine.sock |  0 |         |  3.9 MB |      true |      false |         0 |          0 |                  0 |        |
+-----------------------------------------------+----+---------+---------+-----------+------------+-----------+------------+--------------------+--------+