docs: install instruction

Move the copy/paste instructions to a separate document Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
sergelogvinov · Sep 16, 2024 · 4d79e4e · 4d79e4e
1 parent 5876cd4
commit 4d79e4e
Show file tree

Hide file tree

Showing 3 changed files with 164 additions and 73 deletions.
diff --git a/README.md b/README.md
@@ -12,8 +12,8 @@ Originally, it was designed to work with [Talos CCM](https://github.com/siderola
 The CCM does a few things: it initialises new nodes, applies common labels to them, and removes them when they're deleted. It also supports multiple clusters, meaning you can have one kubernetes cluster across multiple Proxmox clusters.
 
 The basic definitions:
-* kubernetes `region` is a Proxmox cluster `clusters[].region`
-* kubernetes `zone` is a hypervisor host machine name
+* kubernetes label `topology.kubernetes.io/region` is a Proxmox cluster `clusters[].region`
+* kubernetes label `topology.kubernetes.io/zone` is a hypervisor host machine name
 
 This makes it possible for me to use pods affinity/anti-affinity.
 
@@ -24,8 +24,10 @@ This makes it possible for me to use pods affinity/anti-affinity.
 clusters:
   - url: https://cluster-api-1.exmple.com:8006/api2/json
     insecure: false
+    # Proxox auth token
     token_id: "user!token-id"
     token_secret: "secret"
+    # Uniq region name
     region: cluster-1
   - url: https://cluster-api-2.exmple.com:8006/api2/json
     insecure: false
@@ -42,12 +44,18 @@ kind: Node
 metadata:
   labels:
     ...
+    # Type generated base on CPU and RAM
     node.kubernetes.io/instance-type: 2VCPU-2GB
+    # Proxmox cluster name as in the config
     topology.kubernetes.io/region: cluster-1
+    # Proxmox hypervisor host machine name
     topology.kubernetes.io/zone: pve-node-1
   name: worker-1
 spec:
   ...
+  # providerID - magic string:
+  #   cluster-1 - cluster name as in the config
+  #   123 - Proxmox VM ID
   providerID: proxmox://cluster-1/123
 status:
   addresses:
@@ -57,79 +65,19 @@ status:
     type: Hostname
 ```
 
-# Install
+## Install
 
-## Create a token
+See [Install](docs/install.md) for installation instructions.
 
-Official [documentation](https://pve.proxmox.com/wiki/User_Management)
+## Controllers
 
-```shell
-# Create role CCM
-pveum role add CCM -privs "VM.Audit"
-# Create user and grant permissions
-pveum user add kubernetes@pve
-pveum aclmod / -user kubernetes@pve -role CCM
-pveum user token add kubernetes@pve ccm -privsep 0
-```
-
-## Deploy CCM
-
-Create the proxmox credentials
-
-```yaml
-# config.yaml
-clusters:
-  - url: https://cluster-api-1.exmple.com:8006/api2/json
-    insecure: false
-    token_id: "kubernetes@pve!ccm"
-    token_secret: "secret"
-    region: cluster-1
-```
-
-Upload it to the kubernetes:
-
-```shell
-kubectl -n kube-system create secret generic proxmox-cloud-controller-manager --from-file=config.yaml
-```
-
-### Method 1: kubectl
-
-Deploy Proxmox CCM with `cloud-node,cloud-node-lifecycle` controllers
-
-```shell
-kubectl apply -f https://raw.githubusercontent.com/sergelogvinov/proxmox-cloud-controller-manager/main/docs/deploy/cloud-controller-manager.yml
-```
-
-Deploy Proxmox CCM with `cloud-node-lifecycle` controller (for Talos)
-
-```shell
-kubectl apply -f https://raw.githubusercontent.com/sergelogvinov/proxmox-cloud-controller-manager/main/docs/deploy/cloud-controller-manager-talos.yml
-```
-
-### Method 2: helm chart
-
-Create the config file:
-
-```yaml
-# proxmox-ccm.yaml
-config:
-  clusters:
-    - url: https://cluster-api-1.exmple.com:8006/api2/json
-      insecure: false
-      token_id: "kubernetes@pve!ccm"
-      token_secret: "secret"
-      region: cluster-1
-```
-
-Deploy Proxmox CCM
-
-```shell
-helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
-    proxmox-cloud-controller-manager \
-    oci://ghcr.io/sergelogvinov/charts/proxmox-cloud-controller-manager
-```
+Support controllers:
 
-More options you can find [here](charts/proxmox-cloud-controller-manager)
+* cloud-node
+  * Updates node resource.
+  * Assigns labels and taints based on Proxmox VM configuration.
+* cloud-node-lifecycle
+  * Cleans up node resource when Proxmox VM is deleted.
 
 ## Contributing
 

diff --git a/docs/install.md b/docs/install.md
@@ -0,0 +1,143 @@
+# Install
+
+Proxmox Cloud Controller Manager (CCM) supports controllers:
+* cloud-node
+* cloud-node-lifecycle
+
+`cloud-node` - detects new node launched in the cluster and registers them in the cluster.
+Assigns labels and taints based on Proxmox VM configuration.
+
+`cloud-node-lifecycle` - detects node deletion on Proxmox side and removes them from the cluster.
+
+## Create a Proxmox token
+
+Official [documentation](https://pve.proxmox.com/wiki/User_Management)
+
+```shell
+# Create role CCM
+pveum role add CCM -privs "VM.Audit"
+# Create user and grant permissions
+pveum user add kubernetes@pve
+pveum aclmod / -user kubernetes@pve -role CCM
+pveum user token add kubernetes@pve ccm -privsep 0
+```
+
+## Deploy CCM
+
+Create the proxmox credentials config file:
+
+```yaml
+clusters:
+  # List of Proxmox clusters, region mast be unique
+  - url: https://cluster-api-1.exmple.com:8006/api2/json
+    insecure: false
+    token_id: "kubernetes@pve!ccm"
+    # Token from the previous step
+    token_secret: "secret"
+    # Region name, can be any string, it will use as for kubernetes topology.kubernetes.io/region label
+    region: cluster-1
+```
+
+### Method 1: kubectl
+
+Upload it to the kubernetes:
+
+```shell
+kubectl -n kube-system create secret generic proxmox-cloud-controller-manager --from-file=config.yaml
+```
+
+Deploy Proxmox CCM with `cloud-node,cloud-node-lifecycle` controllers
+
+```shell
+kubectl apply -f https://raw.githubusercontent.com/sergelogvinov/proxmox-cloud-controller-manager/main/docs/deploy/cloud-controller-manager.yml
+```
+
+Deploy Proxmox CCM with `cloud-node-lifecycle` controller (for Talos)
+
+```shell
+kubectl apply -f https://raw.githubusercontent.com/sergelogvinov/proxmox-cloud-controller-manager/main/docs/deploy/cloud-controller-manager-talos.yml
+```
+
+### Method 2: helm chart
+
+Create the config file
+
+```yaml
+# proxmox-ccm.yaml
+config:
+  clusters:
+    - url: https://cluster-api-1.exmple.com:8006/api2/json
+      insecure: false
+      token_id: "kubernetes@pve!ccm"
+      token_secret: "secret"
+      region: cluster-1
+```
+
+Deploy Proxmox CCM (deployment mode)
+
+```shell
+helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
+    proxmox-cloud-controller-manager \
+    oci://ghcr.io/sergelogvinov/charts/proxmox-cloud-controller-manager
+```
+
+Deploy Proxmox CCM (daemonset mode)
+
+It makes sense to deploy on all control-plane nodes. Do not forget to set the nodeSelector.
+
+```shell
+helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
+    --set useDaemonSet=true \
+    proxmox-cloud-controller-manager \
+    oci://ghcr.io/sergelogvinov/charts/proxmox-cloud-controller-manager
+```
+
+More options you can find [here](charts/proxmox-cloud-controller-manager)
+
+## Deploy CCM (Rancher)
+
+Official [documentation](https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters)
+
+Rancher RKE2 configuration:
+
+```yaml
+machineGlobalConfig:
+  # Kubelet predefined value --cloud-provider=external
+  cloud-provider-name: external
+  # Disable Rancher CCM
+  disable-cloud-controller: true
+```
+
+Create the helm values file:
+
+```yaml
+# proxmox-ccm.yaml
+config:
+  clusters:
+    - url: https://cluster-api-1.exmple.com:8006/api2/json
+      insecure: false
+      token_id: "kubernetes@pve!ccm"
+      token_secret: "secret"
+      region: cluster-1
+
+# Use host resolv.conf to resolve proxmox connection url
+useDaemonSet: true
+
+# Set nodeSelector in daemonset mode is required
+nodeSelector:
+  node-role.kubernetes.io/control-plane: ""
+```
+
+Deploy Proxmox CCM (daemondset mode)
+
+```shell
+helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
+    proxmox-cloud-controller-manager \
+    oci://ghcr.io/sergelogvinov/charts/proxmox-cloud-controller-manager
+```
+
+## Deploy CCM with load balancer (optional)
+
+This optional setup to improve the Proxmox API availability.
+
+See [load balancer](loadbalancer.md) for installation instructions.
diff --git a/docs/loadbalancer.md b/docs/loadbalancer.md
@@ -41,10 +41,10 @@ kubectl apply -f proxmox-service.yaml
 
 Second, we need to deploy proxmox CCM with sidecar load balancer.
 Haproxy will resolve the `proxmox.kube-system.svc.cluster.local` service and uses IPs from the endpoints to distribute traffic.
-Proxmox CCM will use the `proxmox.domain.com` domain to connect to the proxmox cluster wich is resolved to the load balancer IP (127.0.0.1).
+Proxmox CCM will use the `proxmox.domain.com` domain to connect to the proxmox cluster which is resolved to the load balancer IP (127.0.0.1).
 
 ```yaml
-# Helm Chart values
+# CCM helm chart values
 
 config:
   clusters: