Runbook-playbook-errors-after-upgrade

Issue: After upgrading to Turbine 26.4.3, certificate trust failures prevented Turbine services from connecting to RabbitMQ.

Freshdesk ticket: https://support.swimlane.com/a/tickets/30764

Errors observed:

turbine-api / turbine-content-library:

Failed to connect to AMQP
Error: self-signed certificate in certificate chain
SELF_SIGNED_CERT_IN_CHAIN

swimlane-tasks:

Connection refused (turbine-api.default.svc.cluster.local:3000)
Polly.CircuitBreaker.BrokenCircuitException:
The circuit is now open and is not allowing calls.

Recovery performed:

Use a maintenance window. These commands can regenerate the root CA and restart all application/database workloads in default.

Review the matching Secrets:

kubectl get secret -n default -l controller.cert-manager.io/fao

Delete matching Secrets to trigger reissuance:

kubectl delete secret -n default -l controller.cert-manager.io/fao

Confirm the Secrets are recreated and certificates are Ready before continuing:

kubectl get secret -n default -l controller.cert-manager.io/fao
kubectl get certificates -n default

Restart workloads to reload certificates:

kubectl rollout restart deploy,sts -n default
kubectl get pods -n default -w

Validation:

Check for recurring certificate errors:

kubectl logs -n default deployment/turbine-api -c turbine-api --since=5m | grep -Ei 'self-signed|Failed to connect to AMQP'

Run a test playbook and confirm successful completion.

Result in this incident: All playbooks resumed working after certificate reissuance and workload restarts.