Skip to content

Commit abceea2

Browse files
author
Kaarthikeyan Subramanian
committed
Blocked nodes label action
1 parent e94c519 commit abceea2

File tree

1 file changed

+9
-3
lines changed

1 file changed

+9
-3
lines changed

articles/aks/upgrade-cluster.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -88,16 +88,22 @@ kubectl get nodes --show-labels=true
8888

8989
The blocked nodes are unscheduled for pods and marked with the label `"kubernetes.azure.com/upgrade-status: Quarantined"`. The maximum number of nodes that can be left blocked can't be more than the `Max-Surge` value.
9090

91-
### How do I remove the blocked nodes?
91+
### What action can i do from here on?
9292

93-
First resolve the issue causing the drain. The following example removes the responsible PDB:
93+
First resolve the underlying issue causing the drain. The following example removes the responsible PDB:
9494

9595
```bash
9696
kubectl delete pdb nginx-pdb
9797
poddisruptionbudget.policy "nginx-pdb" deleted.
9898
```
99+
If you are confident the issue is now resolved , then you can go ahead and remove the label `"kubernetes.azure.com/upgrade-status: Quarantined"` placed on undrainable nodes. This can be done as follows:
99100

100-
Then delete the blocked node using the `az aks nodepool delete-machines` command. This command is useful if you intend to reduce the node pool footprint by removing nodes left behind in older versions.
101+
```bash
102+
kubectl label nodes <node-name> <label-key>-
103+
```
104+
Any subsequent 'PUT' operation will attempt to reconcile the 'failed provisioning status' on the cluster to 'success' first. The quarantined nodes shall not be considered for any subsequent put or reconcile. You have to explicitly remove the labels as mentioned previously for any blocked nodes to be considered.
105+
106+
You can also delete the blocked node using the `az aks nodepool delete-machines` command. This command is useful if you intend to reduce the node pool footprint by removing nodes left behind in older versions.
101107

102108
```azurecli-interactive
103109
az aks nodepool delete-machines --cluster-name MyCluster --machine-names aks-nodepool1-test123-vmss000000 --name nodepool1 --resource-group TestRG

0 commit comments

Comments
 (0)