Swimlane 10.20.3 Upgrade Instructions

Use these upgrade instructions to upgrade your instance of Swimlane 10.x. The instructions are divided into subsections within this document, guiding you through the upgrade process based on your specific installation type, whether it is an embedded installation or an existing cluster.

Known Issues

Velero Preflight Check Failing During Online Upgrades

Issue:The Velero Preflight check fails for some online upgrades, potentially blocking the upgrade process.

Workaround:Run the following command to verify the installed Velero version: velero version

If Velero Version is v1.15.2, ignore the preflight check and proceed with the deployment.

If Velero Version is NOT v1.15.2, rerun the following command: curl -sSL https://kurl.sh/swimlane-platform-stable-10-20-3 | sudo bash

Kubernetes Version Preflight Check Is Misleading

Issue:The Kubernetes version preflight check shows a misleading message. The supported range for existing Kubernetes cluster installations of Swimlane is 1.27 - 1.30.

Upgrade Swimlane on Embedded Cluster Installations

Use the following instructions to upgrade this version of Swimlane on an embedded cluster installation.

Prerequisites:

Swimlane version 10.19 or later

If you are not on a version that is >= 10.19, contact your Swimlane support representative.

MongoDB 7.0.17

Important! Swimlane recommends that you perform a MongoDB backup before starting this update. For more information, see Backup and Restore on an Embedded Cluster with Snapshots.

Note: If you are already on version 10.20.2 no infrastructure upgrade is required. Proceed to step 5

If Swimlane uses an external MongoDB, scale down the cluster by setting the replicas for deployments and stateful sets (STS) to 0. Then upgrade your external MongoDB servers to version 7.0.17 and then follow the upgrade instructions.

During the startup of the external MongoDB process, use the following commands:

--tlsAllowConnectionsWithoutCertificates --tlsAllowInvalidCertificates

kubectl scale deploy <deployment name> -n <name space> --replicas=0 kubectl scale sts <sts name> -n <name space> --replicas=0

Upgrade Instructions:

SSH into any node in your deployment and execute the following command to begin the Swimlane Platform Installer component upgrade: $ curl -sSL https://kurl.sh/swimlane-platform-stable-10-20-3 | sudo bash -s ha NOTE: If you have configured SELinux, add this to the end of the command above:installer-spec-file=se.yaml

A Kubernetes upgrade may be required depending on the current version in your environment. For a multi-node (HA) cluster, the script prompts you to drain node one (1) and apply the upgrade. Once the upgrade on node one (1) is complete, the process continues running. It then asks for confirmation to drain node two (2) as well as a command to run in that node.For example: ✔ Kubernetes host packages installed node/ip-10-1-0-147.swimlane.io already uncordoned Drain node ip-10-1-0-186.swimlane.io to prepare for upgrade? (Y/n) y ... pod/swimlane-syslog-receiver-86f79c8fd6-2rvwt evicted pod/swimlane-tasks-5dfd4d5f94-tkrb2 evicted node/ip-10-1-0-186.swimlane.io evicted Run the upgrade script on remote node to proceed: ip-10-1-0-186.swimlane.io curl https://kurl.sh/swimlane-platform-stable-10-20-3/upgrade.sh | sudo bash -s kubernetes-version=1.30.0

Run the upgrade command on the node that was drained. Usekubectl get nodes -o wideto find the IP of the node that was drained.

The primary script that ran in node 1 will detect that node 2 has been upgraded and, and will now ask to drain and upgrade node 3. Follow the same steps for node 2 to complete the kubernetes portion of the upgrade.

Once the additional nodes are complete, press Enter in the terminal window of the first node in which you ran the install command to complete the installation.

Go to the Version History tab and locate the newly published version.If this version doesn’t appear initially, click Check for Updates.

A pop-up window appears, click Go to updated version.

Click Deploy on the new 10.20.3 version with the highest sequence number at the top of the versions list (currently 10.20.3_XXX).

Once the Application tab shows ‘Ready’ your upgrade is complete. See Update the Swimlane Python Driver Python Package if you want to update the driver at this time.

Upgrade Swimlane on Existing Cluster Installations

Note: If you are already on version 10.20.2, no KOTS addon installation is needed, as version 10.20.2 includes the latest KOTS version. Proceed to step 2.

Prerequisites:

Swimlane version 10.19 or later If you are not on a version that is >= 10.19, contact your Swimlane support representative.

MongoDB 7.0.15

Kubernetes 1.27 to 1.30

KOTS kubectl plugin 1.122.0

Velero 1.14.x

Important! Swimlane recommends that you perform a MongoDB backup prior to starting this update. For more information, see Backup and Restore on an Existing Cluster with Snapshots.

Upgrading to Kotsadm v1.124.4:

Download the latest SPI kubectl add-on for the OS version where you run kubectl commands from: $ curl https://kots.io/install/v1.124.4 | bash $ kubectl kots version The KOTS kubectl plugin version should match what is listed in the prerequisites above.

Upgrading to Velero 1.15:

See the Velero documentation for details: https://velero.io/docs/v1.13/upgrade-to-1.13/

Upgrade the Swimlane KOTS application:

Next, upgrade the Swimlane Platform Installer (SPI): $ kubectl kots admin-console upgrade -n <namespace>

Go to the Version History tab and locate the newly published version.If this version doesn’t show up initially, click Check For Updates.

A pop-up window appears, click Go to updated version or visit the Version History tab.

Click Deploy on the new 10.20.3 version with the highest sequence number at the top of the versions list.

Once the Application tab shows Ready your upgrade is complete.

See Update the Swimlane Python Driver Python Package if you want to update the driver at this time.

Upgrade Swimlane on Embedded Airgap Cluster Installations

Note: If you are already on version 10.20.2, there is no need for an infrastructure upgrade. Proceed to step 8.

Prerequisites:

Swimlane version 10.19 or laterIf you are not on a version that is >= 10.19, contact your Swimlane support representative.

MongoDB 7.0.15

Links to the Offline Installer Package and Offline Airgap Bundle If you do not have these linked, contact your Swimlane support representative.

Important! Swimlane recommends that you perform a MongoDB backup prior to starting this update. For more information, see Backup and Restore on an Embedded Cluster with Snapshots.

Upgrade Instructions:

Copy the latest offline installer package to each air-gapped server.Untar the offline installer package on each server. $ tar -xzvf swimlane-platform-10.20.3_XXX.tar.gz

Run the Swimlane Platform Installer update on any node in your deployment: $ cat install.sh | sudo bash -s airgap ha

Once the installation is complete, if you have a multi-node (HA) cluster, the script prompts you to run an additional command on every node: Run this script on node swimlane-2 to load required images before proceeding: cat ./tasks.sh | sudo bash -s load-images Have images been loaded on node swimlane-2? (Y/n)

Run cat ./tasks.sh | sudo bash -s load-images on nodes two (2) and node three (3) before proceeding to run the command on node one (1).This command can be run simultaneously on nodes 2 and 3.Once the load-images script completes on nodes 2 and 3, enter Y in the node 1 terminal session twice.

The install script now runs on node 1 for a few minutes. Once it completes, follow the prompts to run the upgrade script on nodes 2 and 3, similar to this example (the IP address will depend on your deployment): ✔ Kubernetes host packages installed node/ip-10-1-0-147.swimlane.io already uncordoned Drain node ip-10-1-0-186.swimlane.io to prepare for upgrade? (Y/n) y ... pod/swimlane-syslog-receiver-86f79c8fd6-2rvwt evicted pod/swimlane-tasks-5dfd4d5f94-tkrb2 evicted node/ip-10-1-0-186.swimlane.io evicted Run the upgrade script on remote node to proceed: ip-10-1-0-186.swimlane.io cat upgrade.sh | sudo bash -s kubernetes-version=1.30.0 docker-registry-ip=10.96.3.115

When both upgrade scripts are complete, press Enter on node 1 and then upload the airgap bundle to the SPI dashboard.

Go to the Version History tab and click Upload a new version, then browse to the swimlane-platform-10.20.3_XXX.airgap file and click Upload.

The sequence number “_XXX” of the airgap file may be higher than shown if a new sequence has been published.

Once the upload of the airgap completes, refresh the page or switch tabs to see the status of the file extraction.

Go back to the Version History tab and click Deploy on the new 10.20.3 version at the top of the versions list.

Once the Application tab shows ‘Ready’ your upgrade is complete.

See Update the Swimlane Python Driver Python Package if you want to update the driver at this time.

Upgrade Swimlane on Existing Airgap Cluster Installations Prerequisites:

Swimlane version 10.19 or later If you are not on a version that is >= 10.19, contact your Swimlane support representative.

MongoDB 7.0.15

Kubernetes 1.27-1.30

KOTS kubectl plugin 1.122.0

Velero 1.14.x

Important! Swimlane recommends that you perform a MongoDB backup prior to starting this update. For more information, see Backup and Restore on an Existing Cluster with Snapshots.

Upgrading to Kotsadm v1.124.4:

Download the KOTS Kubectl Add-on:Linux: https://github.com/replicatedhq/kots/releases/download/v1.124.4/kots_linux_amd64.tar.gz OSX: https://github.com/replicatedhq/kots/releases/download/v1.124.4/kots_darwin_all.tar.gz

Download the latest SPI offline package: https://github.com/replicatedhq/kots/releases/download/v1.124.4/kotsadm.tar.gz

Copy the KOTS Kubectl Add-on and SPI offline packages from your jumpbox to each node: $ scp kots_<OS version>.tar.gz user@<swimlaneDNS>:/tmp/kots_<OS version>.tar.gz $ scp kotsadm.tar.gz user@<swimlaneDNS>:/tmp/kotsadm.tar.gz

Untar the SPI Kubectl Add-on: $ tar -zxvf kots_<OS version>.tar.gz

Rename or remove the original kubectl-kots file: $ rm /usr/local/bin/kubectl-kots

Rename and move the latest SPI Kubectl Add-on file to/usr/local/bin/kubectl-kots: $ mv kots kubectl-kots $ mv kubectl-kots /usr/local/bin/

Verify version: $ kubectl kots version

Upgrading to Velero 1.15:

See the Velero documention for details. You’ll have to use a jumpbox to download the new images and move them over. https://velero.io/docs/v1.13/upgrade-to-1.13/

Confirm both client and server version are on the same 1.15.0 release:$ ./velero version

Upgrade the Swimlane KOTS application:

Note: If you are already on version 10.20.3, there is no need for an infrastructure upgrade. Proceed to step 3.

Push the SPI (kotsadm) images to your registry: $ kubectl kots admin-console push-images kotsadm.tar.gz <your-registry-endpoint>/<your-registry-name> --registry-username <username> --registry-password <password>

Next, upgrade the Swimlane Platform Installer (SPI): $ kubectl kots admin-console upgrade --kotsadm-registry <your-registry-endpoint>/<your-registry-name> --registry-username <username> --registry-password <password> -n <namespace> When the SPI upgrade is complete, upload the Swimlane airgap bundle to the SPI dashboard.

Go to the Version History tab and click Upload a new version, then browse to the swimlane-platform-10.20.3_XXX.airgap file and click Upload.

The sequence number “_XXX” of the airgap file may be higher than shown if a new sequence has been published

Once the upload of the airgap completes, refresh the page or switch tabs to see the status of the file extraction.

Go to the Version History tab - you will see a version for 10.20.3_XXX.

Click Deploy on the new 10.20.3_XXX version with the highest sequence number at the top of the versions list.

Once the Application tab shows ‘Ready’ your upgrade is complete.

See Update the Swimlane Python Driver Python Package if you want to update the driver at this time.

Update the Swimlane Python Driver Python Package

See Update the Swimlane Python Driver Python Packagehow to update the Swimlane Python Driver package

Download and Push New Velero Images

Use these commands to download and push new Velero images.

$ docker pull velero/velero:v1.15.2 $ docker tag velero/velero:v1.15.2 <my-registry>/<my-namespace>/velero/velero:v1.15.2 $ docker push <my-registry>/<my-namespace>/velero/velero:v1.15.2 $ docker pull velero/velero-restic-restore-helper:v1.15.2 $ docker tag velero/velero-restic-restore-helper:v1.15.2 <my-registry>/<my-namespace>/velero/velero-restic-restore-helper:v1.15.2 $ docker push <my-registry>/<my-namespace>/velero/velero-restic-restore-helper:v1.15.2 $ docker pull velero/velero-plugin-for-aws:v1.11.1 $ docker tag velero/velero-plugin-for-aws:v1.11.1 <my-registry>/<my-namespace>/velero/velero-plugin-for-aws:v1.11.1 $ docker push <my-registry>/<my-namespace>/velero/velero-plugin-for-aws:v1.10.1 $ docker pull velero/velero-plugin-for-gcp:v1.11.1 $ docker tag velero/velero-plugin-for-gcp:v1.11.1 <my-registry>/<my-namespace>/velero/velero-plugin-for-gcp:v1.11.1 $ docker push <my-registry>/<my-namespace>/velero/velero-plugin-for-gcp:v1.11.1 $ docker pull velero/velero-plugin-for-microsoft-azure:v1.11.1 $ docker tag velero/velero-plugin-for-microsoft-azure:v1.11.1 <my-registry>/<my-namespace>/velero/velero-plugin-for-microsoft-azure:v1.11.1 $ docker push <my-registry>/<my-namespace>/velero/velero-plugin-for-microsoft-azure:v1.11.1

Troubleshooting

Upgrading may occasionally leave the Hangfire notifications in a corrupted state, causing the swimlane-api and swimlane-tasks pods to crash loop with logs like the following:

swimlane-api: Hangfire.Mongo.Migration.MongoMigrationException: Migration failed in Hangfire.Mongo.Migration.Steps.Version17.AddNotificationsCollection swimlane-tasks: Core.ErrorHandling.ContractRequiresException: RabbitMQConnectionString is not defined in Orchestration configuration.

Deleting the Hangfire notifications collection in MongoDB resolves this issue.

kubectl exec swimlane-tools-0 -- /bin/bash -c 'mongo $SWIMLANE_Data__Mongo__SwimlaneConnectionString --quiet --tls --tlsAllowInvalidCertificates --eval \'db["hangfire.notifications"].drop()\''