I hope Kubernetes Node Management is a crucial part of a Kubernetes cluster. In terms of node maintenance, we all need to accept that the compute resources such as physical servers or cloud instances need to have a downtime for maintenance works such as hardware changes, Operating system upgrade or any other maintenance work. Same theory applies to the Kubernetes Cluster Nodes and lets see how we can perform the Kubernetes Node Management, Maintenance and editing PODs without interrupting to it’s services.
I have four node Kubernetes cluster on Oracle Ravello Cloud, one Master node and three worker nodes with running Nginx POD with three replicas.
Lets see how we can perform the Cluster Maintenance work without interrupting the running Services
Evict PODs and Safely Perform A Node Maintenance
To view the Kubernetes node status execute the below commands
kubectl get nodes
To check the running POD status and the respective nodes execute below commands
kubectl get pods -o wide
Executing below the command you can safely take a node out from the cluster and running PODs will be evicted from the node
kubectl drain [NODE_HOSTNAME] --ignore-daemonsets
In the below screen capture two Nginx replicas are running in “kube-02.localdomain” node and after executing the command PODs are running in “kube-04.localdomain” node.
Subjected node which is “Kubelet-02.localdomain” is disabled for POD scheduling after performing this command
After verifying the POD status you can bring down the node and perform the maintenance operation now.
Once, the operation completed we can “uncordon” the node and schooling will be enabled again
Complete Operation is explained in the below screen capture
Deleting A Node From The Cluster
If we are planning to delete a node from the Kubernetes cluster, we have to drain, evict the running PODs and disable the POD scheduling first.
To delete the POD below command is used
kubectl delete node [NODE_NAME]
Generate A New Token And Add A New Worker Node
Lets assume that after deleting a node we are planning to add a new node to our Kubernetes Cluster. We might don’t have the “kubeadm join” command and the token with us. We can follow the below procedure to generate a new token and add the worker node to the cluster
To view the token list execute the below command
kubeadm token list
Generate a token with below command, if you don’t have a listed token
kubeadm token generate
Take the generated token and create a command to join the worker node with below command
kubeadm token create [GENERATED_TOKEN] --ttl 23h --print-join-command
Issue this command in the worker node and it will be joining to the cluster. If you need to see how to create a Kubernetes cluster from the scratch read my previous article.