Context
When running Terraform Enterprise Flexible Deployment Options (FDO) in Kubernetes, the application pod's description may show failed readiness probes for a period of time during the startup process.
kubectl describe pod terraform-enterprise-77f744d7b8-4ggnr -n terraform-enterprise
Name: terraform-enterprise-77f744d7b8-4ggnr
Namespace: terraform-enterprise
Priority: 0
Service Account: terraform-enterprise
[[TRIMMED FOR LENGTH]]
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 54m default-scheduler Successfully assigned terraform-enterprise/terraform-enterprise-77f744d7b8-4ggnr to aks-default-14506778-vmss000000
Normal Pulling 54m kubelet Pulling image "images.releases.hashicorp.com/hashicorp/terraform-enterprise:v202312-1"
Normal Pulled 54m kubelet Successfully pulled image "images.releases.hashicorp.com/hashicorp/terraform-enterprise:v202312-1" in 374.473971ms (374.484771ms including waiting)
Normal Created 54m kubelet Created container terraform-enterprise
Normal Started 54m kubelet Started container terraform-enterprise
Warning Unhealthy 54m (x3 over 54m) kubelet Readiness probe failed: Get "http://10.244.1.4:8080/_health_check": dial tcp 10.244.1.4:8080: connect: connection refused
Warning Unhealthy 51m (x18 over 54m) kubelet Readiness probe failed: HTTP probe failed with statuscode: 502
Details
In Kubernetes there are two probe types, readiness and liveness. Liveness probes determine if an application is alive or not, whereas the readiness probe is for determining if an application is ready to receive traffic or not. Kubernetes will prevent routing traffic to services of applications which are not passing the readiness check.
This behavior is expected when the Terraform Enterprise application is started as the startup process is not instant. The failed readiness probes should resolve on their own within a few minutes when the application starts up and is ready to receive traffic, but if the readiness probe does not succeed after a few minutes, this could be an indication of an application error. If the issue does not resolve, please reach out to HashiCorp Support.