Preliminary diagnostic data for Kubernetes troubleshooting

this document explains how to gather preliminary diagnostic data for troubleshooting swimlane platform deployments that include kubernetes clusters to get detailed information about the overall health of your cluster, you can run kubectl cluster info dump | tee cluster info log nodes check if all the nodes are all registered correctly kubectl get nodes o wide | tee nodes log verify that all of the expected nodes are present and that they are all in the ready state describe each node that doesn’t have status equal ready kubectl describe node \<name> | tee \<name> desc log confirm that both the kubelet and docker services are running for services in containerd kubelet; do echo $services ; systemctl is active $services ; systemctl is enabled $services ; echo ""; done if either of the services are inactive, capture the last 3 hours of log journalctl u \<service name> since "3 hour ago" > /tmp/\<service name> log txt pods most importantly, collect information about the health of the pods within the cluster run to capture all recent events kubectl get events field selector type!=normal sort by=' metadata creationtimestamp' all namespaces > /tmp/k8s events log note by default, the amount of time to retain events is 1h run and capture its output kubectl get pods all namespaces o wide | tee all pods log for each pod that’s shown in the get pods output to be in any state other than running/completed run capture its output kubectl logs n \<namepsace name> \<pod name> > /tmp/\<pod name> log txt run capture its output kubectl describe pod n \<namespace name> \<pod name> > /tmp/\<pod name> desc txt since the output from these several commands can be voluminous, copy each command's output into a text file and attach these files to your support portal ticket to minimize the amount of copying and pasting when formulating the above commands, try the following \# save the name of the swimlane namespace into an environment variable export ns=replicated very long app id kubectl get pods n $ns support bundle if your swimlane deployment includes an admin console/dashboard then follow the steps below to generate and submit a support bundle visit https //\<load balancer ip> 8800 browse to the support/troubleshoot section download the support bundle to your workstation attach the support bundle to your support ticket troubleshooting kubectl if the first attempt to invoke kubectl results in this error, the connection to the server localhost 8080 was refused… , then run \# remember to specify the dash when changing user accounts sudo su echo $kubeconfig if the output thereof is empty then export kubeconfig=/etc/kubernetes/admin conf if that works then add this export command to your linux user profile within the master nodes so that it’s invoked whenever you begin an ssh session thank you for helping us to gather this diagnostic data!