Runbook for New Installation of Online HA Turbine with HAProxy

execute these carefully, as they involve enabling selinux and firewalld on the nodes and the haproxy load balancer the following steps are required install rocky linux 9 4 os three nodes for turbine and one node for the haproxy load balancer for rocky linux os and redhat os, the following needs to be installed before installing turbine nfs utils conntrack tools socat git fio containerd for the actual 3 nodes, follow the documentation from the system requirements system requirements for an embedded cluster install https //docs swimlane com/turbine installer/system requirements for an embedded cluster install online embedded cluster install install turbine on an embedded kubernetes cluster https //docs swimlane com/turbine installer/install turbine on an embedded kubernetes cluster component value how to confirm? confirmed node 1 (y/n) cpu 16 cpu cores (typical ha small) $ lscpu grep '^cpu(s) '(should return 16 or more) y cpu instruction set avx required $ lscpu grep e "avx y flags"(should return “avx” or “avx2”) memory 64 gb ram (typical ha small) $ free mh y storage 600 gb ssd / 3000 iops per node (typical) $ df h or df h head n 20 y pods (ha targets) api 3 tasks 3 web 3 y $ kubectl get pods a(after deploy) tenant 3 (odd ≥3 for mongo/postgres) reports 3 mongodb 3 1\ artifacts & secrets to prepare turbine yaml license for the kots admin console ( yaml file) turbine application license for first ui login ( lic file) planned turbine hostname (fqdn / dns name) strong mongo encryption key and mongo password — store in a secure vault (effectively immutable after install) ha load balancer address for the kubernetes api / control plane odd number capacity for replica sets (minimum 3 nodes for mongodb and postgresql for mongodb and postgresql to form a quorum) for the haproxy load balancer, the following specifications should be sufficient cpu 8 vcpus storage 80 gb ssd memory 16–32 gb ram (depending on the resources available in the customer's environment) for the online nodes, the following specifications should be sufficient cpu 16 vcpus storage 700 gb ssd memory 64gb ram (depending on the resources available in the customer's environment) please ensure the following dedicated partitions are configured on each turbine node with the specified storage capacity do not block these partitions from writing into /etc/fstab, otherwise when the node is rebooted, you will lose the partitions also do not enable noexec on any of the /var dorectory in the /etc/fstab /var/lib/kubelet — 150 gb /var/lib/containerd — 150 gb /var/lib/openebs — 300 gb enable selinux and firewalld on the haproxy load balancer node a enable firewalld to start sudo systemctl enable firewalld b ensure firewalld is running sudo firewall cmd state c enable selinux change the status of the service in the /etc/selinux/config file to enforcing run the following command to make sure it is enabled and enforcing sestatus configure firewalld please run the following commands on the haproxy server sudo firewall cmd remove service cockpit sudo firewall cmd remove service dhcpv6 client sudo firewall cmd remove service ssh sudo firewall cmd zone public add port 22/tcp sudo firewall cmd zone public add port 443/tcp sudo firewall cmd zone public add port 6443/tcp sudo firewall cmd zone public add port 8800/tcp sudo firewall cmd runtime to permanent sudo firewall cmd reload these commands modify the firewalld configuration on your linux server they remove some default allowed services and explicitly allow only the ports required for ssh, https, the kubernetes api, and the replicated admin console configure selinux for haproxy please run the below commands on the haproxy server sudo semanage port add type http port t proto tcp 6443 sudo semanage port add type http port t proto tcp 8800 configure haproxy (layer 4 / tcp) turbine does not support a layer 7 load balancer for the kubernetes api, so the api must be tcp mode this example runs everything in tcp mode reference swimlane haproxy load balancer docs https //docs swimlane com/turbine installer/haproxy load balancer 3 1 install sudo dnf install y haproxy # installs haproxy 2 8 x on rocky 9 verify the installation rpm q haproxy nfs utils conntrack tools socat git fio policycoreutils python utils testing connectivity across servers/node for ip in \<node1 ip> \<node2 ip> \<node3 ip>; do echo "testing $ip" nc vz "$ip" 6443 nc vz "$ip" 8800 nc vz "$ip" 443 done on each online node, check firewall cmd state firewall cmd list all allow the required ports firewall cmd permanent add port=443/tcp firewall cmd permanent add port=6443/tcp firewall cmd permanent add port=8800/tcp firewall cmd reload retest from the haproxy server for ip in \<node1 ip> \<node2 ip> \<node3 ip>; do echo "testing $ip" nc vz "$ip" 6443 nc vz "$ip" 8800 nc vz "$ip" 443 done after opening the ports, these are the possible results connection succeeded — service is running and reachable connection refused — firewall is no longer blocking, but no service is listening yet no route to host — firewall or network rejection still exists if kubernetes/turbine is not installed yet, connection refused is expected until those services start note now add your haproxy config file validate it haproxy c f /etc/haproxy/haproxy cfg expected configuration file is valid then restart haproxy systemctl restart haproxy systemctl status haproxy configure haproxy similarly as per the following doc https //docs swimlane com/turbine installer/haproxy load balancer https //docs swimlane com/turbine installer/haproxy load balancer example configuration for a layer 4 haproxy server \################## # global options # ################## global defaults timeout client 30s timeout server 30s timeout connect 30s listen stats bind 8080 mode http stats enable stats uri / stats hide version ## # turbine frontend / backend ## frontend turbine frontend mode tcp bind 80 # optional https redirection bind 443 http request redirect scheme https unless { ssl fc } # optional https redirection default backend turbine backend backend turbine backend mode tcp balance roundrobin option tcp check server tpi node 1 tpi node 1 swimlane io 443 check server tpi node 2 tpi node 2 swimlane io 443 check server tpi node 3 tpi node 3 swimlane io 443 check ## # turbine platform installer ui frontend / backend ## frontend replicated frontend mode tcp bind 8800 default backend replicated backend backend replicated backend mode tcp balance roundrobin option tcp check server tpi node 1 tpi node 1 swimlane io 8800 check server tpi node 2 tpi node 2 swimlane io 8800 check server tpi node 3 tpi node 3 swimlane io 8800 check ## # kubernetes api frontend / backend ## frontend kube api frontend mode tcp bind 6443 default backend kube api backend backend kube api backend mode tcp balance roundrobin option tcp check server tpi node 1 tpi node 1 swimlane io 6443 check server tpi node 2 tpi node 2 swimlane io 6443 check server tpi node 3 tpi node 3 swimlane io 6443 check then run service haproxy reload to restart the haproxy service haproxy c f /etc/haproxy/haproxy cfg systemctl enable haproxy systemctl start haproxy systemctl status haproxy set ip forwarding on all three nodes for a kubernetes/turbine cluster, net ipv4 ip forward should be 1 to check sysctl net ipv4 ip forward make it persistent across reboots cat <\<eof | sudo tee /etc/sysctl d/99 kubernetes conf net ipv4 ip forward = 1 eof apply it immediately sudo sysctl system add required ports to selinux on all 3 nodes semanage port add type http port t proto tcp 2379; semanage port add type http port t proto tcp 2380; semanage port add type http port t proto tcp 6443; semanage port add type http port t proto udp 8472; semanage port add type http port t proto tcp 8800; semanage port add type http port t proto tcp 10250; external zone, firewalld config on all 3 nodes firewall cmd zone external add masquerade; public zone, firewalld config on all 3 nodes firewall cmd zone public remove service cockpit; firewall cmd zone public remove service dhcpv6 client; firewall cmd zone public remove service ssh; firewall cmd zone public add port 22/tcp; firewall cmd zone public add port 443/tcp; firewall cmd zone public add port 2379/tcp; firewall cmd zone public add port 2380/tcp; firewall cmd zone public add port 6443/tcp; firewall cmd zone public add port 8472/udp; firewall cmd zone public add port 8800/tcp; firewall cmd zone public add port 10250/tcp; trusted zone, firewalld config on all 3 nodes firewall cmd zone trusted add interface cni0; firewall cmd zone trusted add interface flannel 1; firewall cmd zone trusted add interface kube ipvs0; firewall cmd zone trusted add port 2379/tcp; firewall cmd zone trusted add port 2380/tcp; firewall cmd zone trusted add port 4789/udp; firewall cmd zone trusted add port 5000/tcp; firewall cmd zone trusted add port 6783/tcp; firewall cmd zone trusted add port 6783/udp; firewall cmd zone trusted add port 6784/udp; firewall cmd zone trusted add port 8080/tcp; firewall cmd zone trusted add port 8472/udp; firewall cmd zone trusted add port 10250/tcp; firewall cmd zone trusted add port 10257/tcp; firewall cmd zone trusted add port 10259/tcp; save and reload configured firewalld rules on all 3 nodes firewall cmd runtime to permanent; firewall cmd reload; test connectivity between the three nodes and the haproxy on port 6443 test tcp connectivity from each node to haproxy run this on each node nc vz \<haproxy ip> 6443 curl k v \<haproxy url> 6443 ensure this works before attempting the turbine installation; otherwise, the install will fail create a patch yaml file to include firewall and selinux apiversion "cluster kurl sh/v1beta1" kind "installer" metadata name "patch" spec kubernetes hacluster true loadbalanceraddress "haproxy iq ts swimlane us 6443" firewalldconfig firewalld "enabled" bypassfirewalldwarning true disablefirewalld false hardfailonfirewalld false preserveconfig false selinuxconfig selinux "enforcing" type "targeted" preserveconfig false disableselinux false install step 1 log in to node 1 from a command line interface, log in as a privileged user to the system that will host the initial turbine platform instance ssh \<privileged user>@\<node1 ip address> step 2 start the ha installation on node 1 run the following command curl ssl https //kurl sh/turbine stable multitenant | sudo bash s ha installer spec file=patch yaml step 3 accept the installation permissions during the installation, the installer will request confirmation before proceeding when prompted, type y press enter to continue allow the installation to complete do not interrupt the process or close the terminal session while it is running the script will complete on node 1 and generate the join commands ensure that you use only the control plane (master) join command on the remaining two nodes so they are added as control plane nodes step 4 record the generated join commands after the installation completes successfully on node 1, the installer will display cluster join commands locate and securely copy the command labeled master join important use only the master join command for node 2 and node 3 do not use the worker join command the join command contains sensitive cluster information and should not be shared or stored in an unsecured location step 5 join node 2 as a master node log in to node 2 ssh \<privileged user>@\<node2 ip address> run the exact master join command generated by node 1 curl fssl https //kurl sh/version/v2026 07 08 0/turbine stable multitenant/join sh | sudo bash s kubernetes master address=10 33 102 x 6443 kubeadm token=\<fresh> kubeadm token ca hash=sha256 \<fresh> kubernetes version=1 35 4 cert key=\<fresh> control plane ekco address=10 33 102 x 31880 ekco auth token=\<fresh> docker registry ip=10 96 1 x additional no proxy addresses=10 96 0 0/22,10 32 0 0/20 primary host=10 33 102 x installer spec file=patch yaml add the installer spec file=patch yaml while running on the other nodes and follow the prompts if the command requests confirmation, type y press enter and wait for the node join operation to finish successfully step 6 join node 3 as a master node log in to node 3 ssh \<privileged user>@\<node3 ip address> run the same master join command generated by node 1 curl fssl https //kurl sh/version/v2026 07 08 0/turbine stable multitenant/join sh | sudo bash s kubernetes master address=10 33 102 x 6443 kubeadm token=\<fresh> kubeadm token ca hash=sha256 \<fresh> kubernetes version=1 35 4 cert key=\<fresh> control plane ekco address=10 33 102 x 31880 ekco auth token=\<fresh> docker registry ip=10 96 1 x additional no proxy addresses=10 96 0 0/22,10 32 0 0/20 primary host=10 33 102 x installer spec file=patch yaml if the command requests confirmation, type y press enter and wait for the node join operation to finish successfully once the install completes, save the url and password that the install generates you'll also receive the kots admin console (kotsadm) login credentials, including the initial password please make sure to save these credentials, as the password is displayed only once during installation you can change it later if needed also, run the following on node 1 to start using the kubectl commands mkdir p $home/ kube sudo cp i /etc/kubernetes/admin conf $home/ kube/config sudo chown $(id u) $(id g) $home/ kube/config kubectl get pods a step 7 verify the cluster after node 2 and node 3 have joined, return to node 1 and verify the kubernetes nodes kubectl get nodes o wide confirm that all three nodes are listed all three nodes show a ready status no node remains in a notready state example expected result name status roles age version node1 ready control plane node2 ready control plane node3 ready control plane check the status of all pods kubectl get pods a allow sufficient time for the pods to initialize verify that the required pods eventually reach a healthy state such as running or completed log in to the turbine admin console you saved it earlier http //10 20 36 106 8800 http //10 20 36 106 8800 click on continue click on advanced continue with the self signed cert you can also add your load balancer fqdn upload the license file ( yaml) that you received from the swimlane support for any version note the license file should be pinned to stable multitenant on the replicated portal once you have uploaded the license file, click on download turbine from the internet enter your hostname enter your database encryption key/password note pls remember this key/password since it can’t be changed further enable ha and enter default values save the config and deploy note ignore the warning if you have sufficient storage turbine deploying in progress last window of deployment status from the admin console you can also verify from the nodes kubectl get pods a o wide confirm that all three nodes are listed all three nodes show a ready status no node remains in a notready state all the pods should be up and running expect few in completed state now log in to the turbine ui https //haproxy iq ts swimlane us/ https //haproxy iq ts swimlane us/ upload the turbine ui license file ( lic) continue and create your admin credentials and log in end of article