System Requirements for an Embedded Cluster Install
ο»ΏWhile the Turbine platform can be installed on a single node for testing, a 3+ node cluster is recommended for production environments to provide redundancy and high availability (HA). Any multiple node cluster must have an odd number of total nodes. In embedded HA deployments, stateful services (MongoDB, Postgres, Elasticsearch, and RabbitMQ) remain at 3 replicas even when you add more Kubernetes nodes. Additional nodes increase scheduling capacity for stateless pods, not stateful replicas.
This table details the recommended sizing and data per node:
Components | Single Node | 3 Node Cluster - Small | 3 Node Cluster - Medium | 3 Node Cluster - Large |
|---|---|---|---|---|
CPU | 8 CPU cores | 16 CPU cores | 24 CPU cores | 32 CPU cores |
CPU Instruction Set | AVX required | AVX required | AVX required | AVX required |
MEMORY | 32 GB RAM | 64 GB RAM | 96 GB RAM | 128 GB RAM |
STORAGE | 600 GB SSD / 3000 IOPS per node | 600 GB SSD / 3000 IOPS per node | 1 TB SSD / 3000 IOPS per node | 1 TB SSD / 3000 IOPS per node |
Record Creation Boundaries + Active Users | Records created in a day: 250,000 Total records: 5 million Active users: 10 | Records created in a day: 500,000 Total records: 20 million Active users: 30 | Records created in a day: 1 million Total records: 20 million Active users: 50 | Records created in a day: 1 million Total records: 20 million Active users: 200 |
Integration Calculations | Integrations in use < 20 average Integration actions/day < 250,000 | Integrations in use < 20 average Integration actions/day < 500,000 | Integrations in use < 20 average Integration actions/day < 1 million | Integrations in use > 20 average Integration actions/day < 1 million |
Pods | API: 1 Tasks: 1 Web: 1 MongoDB: 1 Reports: 1 | API: 3 Tasks: 3 Web: 3 MongoDB: 3 Reports: 3 | API: 3 Tasks: 3 Web: 3 MongoDB: 3 Reports: 3 | API: 6 Tasks: 9 Web: 3 MongoDB: 3 Reports: 3 |
External MongoDB Resource Recommendations
This table illustrates the resource recommendations (per node) for a standalone mongo deployment. All of these values can be subtracted from the system requirements above when allocating resources for the remainder of the Turbine pods. For more information about deploying on an External MongoDB cluster, see Deploy Turbine with an External MongoDB ClusterDeploy Turbine with an External MongoDB Cluster.
Components | Single Node | 3 Node Cluster - Sm | 3 Node Cluster - Med | 3 Node Cluster - Lg |
|---|---|---|---|---|
CPU | 4 CPU Cores | 4 CPU Cores | 8 CPU Cores | 8 CPU Cores |
Ram | 16 GB RAM | 16 GB RAM | 16 GB RAM | 32 GB RAM |
Storage | 300 GB SSD / 3000 IOPS per node | 300 GB SSD / 3000 IOPS per node | 700 GB SSD / 3000 IOPS per node | 700 GB SSD / 3000 IOPS per node |
Remaining Turbine Cluster Resources
This table illustrates the resources necessary for the remainder of Turbine if you are using external MongoDB resources:
Components | Single Node | 3 Node Cluster - Sm | 3 Node Cluster - Med | 3 Node Cluster - Lg |
|---|---|---|---|---|
CPU | 4 CPU Cores | 4 CPU Cores | 8 CPU Cores | 24 CPU Cores |
Ram | 16 GB RAM | 16 GB RAM | 48 GB RAM | 96 GB RAM |
Storage | 300 GB SSD / 3000 IOPS per node | 300 GB SSD / 3000 IOPS per node | 300 GB SSD / 3000 IOPS per node | 300 GB SSD / 3000 IOPS per node |
Resource Utilization Thresholds
All nodes need to stay below certain resource utilization thresholds to ensure that pods always have available resources to operate in. If any of the following thresholds are exceeded on a node, all pods on that node will be removed until the resource utilization is addressed:
Resource | Threshold |
|---|---|
Memory | Less than 100 Mebibytes (MiB) Available |
Disk Space (/var/lib/containerd partition) | Less than 15% Available |
Disk Space (/var/lib/kubelet partition) | Less than 10% Available |
Disk Inodes (/var/lib/kubelet partition) | Less than 5% Available |
Backup Requirements
Taking a snapshot requires enough free disk space for a compressed archive of the Swimlane database to be saved in ephemeral storage before it is uploaded to the snapshot destination. Free disk space on the cluster at /var/lib/kubelet should be greater than or equal to the size of the uncompressed database to ensure there is no disk pressure during the snapshot process.
Cloud Service Sizing Tools
Sizing Calculators | Sizing Guides |
|---|---|
Instance Sizing Calculators AWS | ο»ΏInstance Sizing AWSο»Ώ |
Instance Sizing Calculators Azure | ο»ΏInstance Sizing Azureο»Ώ |
Instance Sizing Calculators GCP | ο»ΏInstance Sizing GCPο»Ώ |
Most cloud providers limit IOPS for disks and for instances/virtual machines. Consult your provider's documentation to ensure the effective IOPS for the cluster nodes meet the requirements in the sizing table above.
Provider | Documentation |
|---|---|
AWS | ο»ΏDisk Performance Limitsο»Ώ |
AWS | |
Azure | ο»ΏDisk Performance Limitsο»Ώ |
Azure | ο»ΏVM Performance Limitsο»Ώ |
GCP | ο»ΏDisk Performance Limitsο»Ώ |
GCP | ο»ΏVM Performance Limitsο»Ώ |
Critical Prerequisites
The following operating systems have been tested by Swimlane for your node setup:
- RHEL 9: 9.2, 9.4, 9.5, 9.6 (containerd must be installed before the Swimlane installation)
- Rocky Linux 9: 9.2, 9.4
- Ubuntu: 20.04.6 LTS, 22.04.2 LTS, 24.04.1 LTS
- Ubuntu 20.04 has reached end of life. After the 25.4.0 release, Swimlane Turbine will no longer support deployments running on Ubuntu 20.04. We recommend upgrading to the latest supported Ubuntu version.
- Amazon Linux: Amazon Linux 2, Amazon Linux 2023
Limitations:
- Only Static IPs are supported (dynamic IPs are not allowed), and the selected Static IP cannot be changed later.
- Node hostnames must be static and cannot be changed.
- Nodes must not have any existing installations of Kubernetes, Docker, or Containerd.
- Automatic updates for Kubernetes and Docker-related packages must be disabled. Updating these will be handled through the Turbine Platform Installer.
- Ubuntu 24.04 and Amazon Linux 2023 require that containerd v1.7 to be preinstalled on the host (especially for airβgapped clusters): https://kurl.sh/docs/install-with-kurl/system-requirements#supported-operating-systems.ο»Ώ
- IPv6 and dual-stack networks are not supported.
- IP forwarding must be enabled. The installer enables IPv4 forwarding, but you must ensure it remains enabled on all cluster nodes permanently, including after reboots.
Prerequisites for Installation on Rocky Linux or RHEL 9.x
- nfs-utils
- conntrack-tools
- socat
- git
- fio
Use the following command to install:
sudo dnf -y install nfs-utils conntrack-tools socat git fio container-selinux tar zip unzip
For Rocky Linux 9.2, run the following command to change the file permissions:
sudo chmod 755 /etc/rc.d/rc.local
For each node, ensure that you have:Β
- Sudo/Root access
- NUMA (non-uniform memory access) disabled.
- Accurate system time.
To maintain accurate system time you must have Network Time Protocol (NTP) or a similar time-sync service.
Critical CPUΒ Architecture Prerequisites
CPU architecture must be compatible with MongoDB, unless an external MongoDB solution is used. See the MongoDB Platform Support Matrix for more information.
β οΈ Warning: The AVX CPU instruction set is required for all Turbine deployments using the bundled MongoDB (v5.0+). If AVX is not supported by your CPU or VM configuration, MongoDB will fail to start with an Illegal instruction (core dumped) error and the mongo-0 pod will be stuck in Init:2/3 state. Deployment will not complete.
This requirement does not apply if you are using an external MongoDB solution.
Key requirements:
- AVX (Advanced Vector Extensions) CPU instruction set is required to run MongoDB.
- NUMA (non-uniform memory access) needs to be disabled.
Pre-Installation Verification
Before installing, verify AVX support on each node:
- If the output contains avx, the CPU is compatible.
- If there is no output, the CPU does not support AVX and Turbine deployment will fail.
Virtualized Environment β VM CPU Type Compatibility
If deploying in a virtualized environment, ensure the VM CPU type exposes AVX instructions to the guest OS. Use the table below to check compatibility:
VM CPU Type | AVX Support | Turbine Compatible |
|---|---|---|
x86-64-v2 | No | No β deployment will fail |
x86-64-v3 | Yes (AVX + AVX2) | Yes |
x86-64-v4 | Yes | Yes |
host (passthrough) | Yes (if physical CPU supports AVX) | Yes |
Hypervisor-specific guidance:
- PROXMOX: Set the VM CPU type to x86-64-v3 or higher, or use host passthrough mode.
- VMware ESXi: Ensure the VM hardware version and CPU compatibility mode expose AVX instructions to the guest. Check VM settings β CPU β Expose hardware-assisted virtualization.
- Hyper-V: Use a VM configuration version that supports AVX passthrough. Verify CPU compatibility settings in the VM properties.
- KVM/QEMU: Use -cpu host or a CPU model that includes AVX (e.g., SandyBridge or later).
Critical Network Prerequisites
- The IP ranges 10.32.0.0/20and 10.96.0.0/22 are the default IP ranges used internally in the cluster for Kubernetes service and pod networking. If these are in use in your network and routable by the cluster nodes, the internal cluster IP ranges will need to be overridden. See Define Custom Pod and Service SubnetsDefine Custom Pod and Service Subnets for instructions on overriding the internal cluster IP ranges.
- At a minimum, a network throughput of 1Gbps is generally acceptable for most use-cases. Swimlane recommends, maintaining a latency of less than 5 milliseconds between workload pods and the primary MongoDB pod. For latency between MongoDB replica set members, a more relaxed threshold of 10 milliseconds is sufficient.
- Ensure all nodes are in the same cloud provider region or physical data center network. Nodes behind different WAN links in the same cluster are not supported.
Critical Prerequisites for Airgapped Network Installations
For access to these optional services, ensure you have these things within the airgapped network:
- LDAP login functionality requires an LDAP server inside of the airgapped subnet, or access to outside subnets or the IP or Domain where the LDAP server resides. In order to open these ports, non-secure LDAP uses port 389, but LDAPS (Secure LDAP) uses port 636 and is preferred.
- SSO login functionality requires the service to be able to reach inside the airgapped network (where the Turbine instance resides).
- Email functionality requires the Turbine instance to use a functioning mail proxy that resides within the airgapped subnet, access to outside subnets where an email server resides, or to your chosen email server on the internet. In order to open these ports, non-secure SMTP uses port 25, but the Secure SMTP uses port 587 and is preferred.
- In order to provide threat intelligence enrichment, the SOC Solution requires access to the following Threat Intelligence URLs:
- VirusTotal - https://virustotal.com and subdomains
- URLΒ Haus - URLhaus - Malware URL exchange - and subdomains
- Recorded Future - Recorded Future: Securing Our World With Intelligence and subdomains
- IPQualityScore - Fraud Prevention | Bot Detection | Bot Protection | Prevent Fraud with IPQS and subdomains
- Each of these requires TCP port 443 access and TCP port 80 access.
Critical Load Balancer Prerequisites
A load balancer that supports hairpinning is REQUIRED for HA installations.
Here are some suggested load balancers and configurations that you can use:
- Layer 7 Load Balancers:
- ο»ΏAWS Application Load BalancerAWS Application Load Balancerο»Ώ
- ο»ΏAzure Standard Application GatewayAzure Standard Application Gatewayο»Ώ
- ο»ΏAzure Standard_v2 Application GatewayAzure Standard_v2 Application Gatewayο»Ώ
- ο»ΏGCP HTTPS Load BalancerGCP HTTPS Load Balancerο»Ώ
- ο»ΏHA ProxyHA Proxyο»Ώ
- TCP Forwarding Layer 4 Load Balancers:
- ο»ΏAWS Network Load BalancerAWS Network Load Balancerο»Ώ
- ο»ΏAzure Standard Load BalancerAzure Standard Load Balancerο»Ώ
- ο»ΏGCP TCP Load BalancerGCP TCP Load Balancerο»Ώ
- ο»ΏHA ProxyHA Proxyο»Ώ
Extra Considerations for Load Balancer Set Up on Nodes
Each node in the cluster needs to be reachable by the load balancer on these ports:
- TCP 443 (Turbine Platform UI)
- TCP 8800 (Turbine Platform Installer UI)
- TCP 6443 (Kubernetes API)
- TCP 80 (Optional - HTTP to HTTPS redirect for the Turbine Platform UI)
A load balancer is required for the Kubernetes API and the Turbine Platform Installer.
If you are using a Layer 4 load balancer for the Turbine Platform Installer and the Turbine platform, it can be combined with the Kubernetes API load balancer so that only one is required.
The Kubernetes API load balancer must be a Layer 4 load balancer.
- The front-end port on the load balancer should be 6443 (Kubernetes Control Plane).
- The back-end port on the load balancer to the nodes should be 6443.
The Turbine Platform Installer and the Turbine platform load balancer can be either a Layer 4 or Layer 7 load balancer.
- Front-end ports on the load balancer should be 443 (Turbine platform) and 8800 (Turbine Platform Installer).
- Backend ports on the load balancer map different based on load balancer type
- For Layer 7 load balancers:
- 443 should map to 4443
- 8800 should map to 8800
- For Layer 4 load balancers:
- 443 should map to 443
- 8800 should map to 8800
Other Critical Prerequisites
- For production environments use an Amazon S3 bucket or S3-compatible storage solution to store snapshots externally so that a total cluster outage does not result in a loss of snapshots.
- If Amazon S3 is unavailable for use then an S3-compatible solution like min.io can be used but should be installed external to the cluster.
- In order to maximize disk space, Swimlane recommends that you set up partitions. Here are the minimum recommended partition sizes:
Partition | Size | Description of highest storage consumers by function: | Storage concerns and what to look out for: | ο»Ώ |
|---|---|---|---|---|
/ | 50 GB | Base OS, installation files, logs, and other smaller cluster dependencies | Local logs storage and log rotation policies | ο»Ώ |
/var/lib/containerd | 100 GB | Container images, logs, and runtime volumes | Image growth (unused images get pruned when the default images threshold of 15% is reached) and container log growth | ο»Ώ |
/var/lib/kubelet | 100 GB | Kubernetes runtime, ephemeral storage, and in flight snapshot temporary storage | Snapshot size and pod temporary storage increasing | ο»Ώ |
/var/openebs | 300 GB | Persistent storage subsystem - used for database volumes Swimlane Platform Installer database, and completed local snapshots | ο»Ώ | ο»Ώ |
/var/log | 5 GB | Storage for the Kubernetes API server logs | 100% usage of /var/log/apiserver will cause a Swimlane outage until space is retrieved | Β |
Local storage of snapshots (see note below)
Amount of Turbine data and integrations
The directory /var/lib/kurl/ is the default location for installer temporary files but the path can be overridden at install time. The partition that path is on needs to have 20GB available during the install to ensure successful installation and can be removed after the installation has completed.
Production environments should be using external storage locations for snapshots. If you plan to store snapshots locally in a test/lab environment, you will need to make the /var/openebs partition larger to accommodate them.
If you plan to use a separate disk or volume for /var/openebs ensure it meets the minimum IOPS defined in the table with recommended data and sizing at the beginning of this topic.
The partition /var/log needs at least 5GB available to avoid degrading a Turbine cluster or outage.
Optional: Swimlane does not provide native, application-level encryption for data at rest. Instead it is recommended using disk-level encryption to secure stored data. Most cloud providers and Kubernetes service providers offer options to create encrypted disks or storage classes. For on-premise environments, disk encryption can be configured using standard tools such as dm-crypt.
Note that the configuration and management of disk encryption technologies (for example, dm-crypt) are outside the scope of Swimlane documentation and support.
Domain Name System (DNS)
Use the IP or DNS record for the Kubernetes API load balancer to specify the load balancer address during the initial installation.
A DNS record should be created that points to the Turbine Platform Installer and Turbine platform load balancer. This will be the DNS address specified when you configure Turbine after the installation. Append that address with :8800 in order to access the Turbine Platform Installer UI.
If only one Layer 4 load balancer is used then a single DNS record can be created and used for both purposes.
You must use DNS-compliant records. A DNS record can be up to 63-characters long and can only contain letters, numbers, and hyphens. The record cannot start or end with a hyphen, nor have consecutive hyphens.
Exceptions to Network Access Control Lists (ACL)
Turbine installations on servers with tight Network Access Control (NAC) will need several exceptions made to properly install, license, and stage a Turbine deployment with the Turbine Platform Installer. See the tables below for the Outbound and Inbound exceptions.
Required Outbound URL ACL Exceptions
ο»Ώ
Exception | Purpose |
|---|---|
get.swimlane.io | Turbine platform installation script |
k8s.kurl.sh | Turbine platform installation script |
kurl.sh | Turbine platform installation script |
kurl-sh.s3.amazonaws.com | Turbine platform installation script dependencies |
registry.replicated.com | Turbine platform container images |
proxy.replicated.com | Turbine platform container images |
ghcr.io | Swimlane platform container dependency images |
registry.k8s.io | Swimlane platform container dependency images |
k8s.gcr.io | Turbine platform container dependency images |
storage.googleapis.com | Turbine platform container dependency images |
quay.io cdn.quay.io cdn01.quay.io cdn02.quay.io cdn03.quay.io cdn04.quay.io cdn05.quay.io cdn06.quay.io | Turbine platform container dependency images |
replicated.app | Turbine Platform Installer license verification |
auth.docker.io | Docker authentication |
registry-1.docker.io | Docker registry |
production.cloudflare.docker.com | Docker infrastructure |
files.pythonhosted.org | Python packages for Turbine integrations |
pypi.org | Python packages for Turbine integrations |
<LoadBalancerIP>:6443 | Kubernetes API |
Port Requirements
External Access
The following ports are required externally to allow access to the Turbine platform components:
Ports | Protocol | Purpose | Access From |
|---|---|---|---|
443 | TCP | Turbine platform UI | Clients that need to access the Turbine platform UI |
80 | TCP | Turbine platform UI | Optional - HTTP to HTTPS redirect for the Turbine platform UI |
8800 | TCP | Turbine Platform Installer UI | Clients that need to access the Turbine Platform Installer UI |
22 | TCP | Shell access | Management workstations to manage the cluster nodes and install Turbine |
Between Cluster Nodes
The following ports are required between the clusters nodes to allow cluster operation:
Ports | Protocol | Purpose |
|---|---|---|
2379-2380 | TCP | Kubernetes etcd |
6443 | TCP | Kubernetes API |
8472 | UDP | Kubernetes CNI |
10250-10252 | TCP | Kubernetes components (kubelet, kube-scheduler, kube-controller) |
From Load Balancers
The following ports are required between the cluster nodes and the load balancer(s):
Ports | Protocol | Purpose |
|---|---|---|
443 | TCP | Turbine platform UI |
80 | TCP | Optional - HTTP to HTTPS redirect for the Turbine platform UI |
8800 | TCP | Turbine Platform Installer UI |
6443 | TCP | Kubernetes API |
Available Ports
In addition to all ports listed above in the Between Cluster Nodes table, the following ports are required to be available and unused by other processes on each cluster node in order to install:
Ports | Protocol | Purpose |
|---|---|---|
2381 | TCP | Kubernetes etcd |
8472 | TCP | Kubernetes CNI |
10248 | TCP | Kubernetes kubelet health server |
10249 | TCP | Kubernetes kube-proxy metrics server |
10257 | TCP | Kubernetes kube-controller-manager health server |
10259 | TCP | Kubernetes kube-scheduler health server |
External Monitoring Considerations
Swimlane recommends that any TPI installation has some amount of external monitoring set up and set to alert when any user-defined thresholds are met as to possible instances of your production instances going down. Different installation scenarios may call for different metrics to monitor for, so your implementation may vary. As a baseline, the following metrics are recommended:
- Are all nodes healthy?
- Does each node have sufficient free space in all partitions?
- Are CPU and memory usage levels within acceptable levels?
- Is disk latency within acceptable ranges?
- Are any pods in a not-ready state?
- Are any deployments or StatefulSets not reporting the correct number of ready replicas?
- Do any load balancers have health checks in place? Are they healthy?
Third Party Monitoring Solutions
There are several third-party monitoring solutions that you can use to monitor resource usage for a node. Any tool that you put into use should be installed externally to the cluster so as to not interfere with cluster operations and to be able to alert if a metric enters into a failing scenario. These products may require that their own agents or exporters be installed on the nodes in order to facilitate monitoring. Any agent or exporter should be tested against your cluster to validate that they do not interfere with Turbine operations or port requirements.
The use of .NET Core process monitors such as Dynatrace is known to cause instabilities in Swimlane/Turbine Kubernetes pods, specifically higher than normal CPU and/or memory consumption. Please use such software with caution and uninstall it from your Swimlane/Turbine servers if its use causes Swimlane/Turbine pods to crash and restart.