Introduction
When you start the Terraform Enterprise container, it automatically launches all Terraform Enterprise processes. If there’s a configuration error, the container will crash, making it difficult to troubleshoot since some logs become inaccessible.
Expected Outcome
The steps below allow you to start the Terraform Enterprise container without automatically initiating its processes. This approach lets you manually start the process, so even if a process fails, the container itself remains running, allowing full access to all logs.
Prerequisites
- Terraform Enterprise FDO (Flexible Deployment Option) as documented here
Docker
- Alter the compose.yaml file by adding a different entrypoint
version: "3.9"
name: terraform-enterprise
services:
tfe:
image: images.releases.hashicorp.com/hashicorp/terraform-enterprise:v202409-3
entrypoint: ["sleep", "infinity"]
- Start the container
docker compose up --detach
- Connect to the container
docker exec -it terraform-enterprise-tfe-1 bash
- Start the process that will start the TFE application components
/usr/local/bin/supervisord-run
- If things crash you can review all the logs
terraform-enterprise@d4d1d593bcdc:/var/run$ ls -al /var/log/terraform-enterprise/
total 96
drwxrwxr-x. 2 root root 460 Oct 31 09:42 .
drwxr-xr-x. 1 root root 36 Sep 25 22:18 ..
-rw-r--r--. 1 terraform-enterprise root 1737 Oct 31 09:43 archivist.log
-rw-r--r--. 1 terraform-enterprise root 1673 Oct 31 09:43 atlas.log
-rw-r--r--. 1 terraform-enterprise root 1519 Oct 31 09:43 backup-restore.log
-rw-r--r--. 1 terraform-enterprise root 10036 Oct 31 09:43 licensing.log
-rw-r--r--. 1 terraform-enterprise root 774 Oct 31 09:43 metrics.log
-rw-r--r--. 1 nginx nginx 0 Oct 31 09:42 nginx-access.log
-rw-r--r--. 1 nginx nginx 0 Oct 31 09:42 nginx-error.log
-rw-r--r--. 1 terraform-enterprise root 0 Oct 31 09:42 nginx.log
-rw-r--r--. 1 terraform-enterprise root 215 Oct 31 09:43 outbound-http-proxy.log
-rw-r--r--. 1 terraform-enterprise root 2749 Oct 31 09:43 postgres.log
-rw-r--r--. 1 terraform-enterprise root 1616 Oct 31 09:43 redis.log
-rw-r--r--. 1 terraform-enterprise root 1978 Oct 31 09:43 sidekiq.log
-rw-r--r--. 1 terraform-enterprise root 966 Oct 31 09:43 slug-ingress.log
-rw-r--r--. 1 terraform-enterprise root 6133 Oct 31 09:43 supervisord.log
-rw-r--r--. 1 terraform-enterprise root 2125 Oct 31 09:43 task-worker.log
-rw-r--r--. 1 terraform-enterprise root 10808 Oct 31 09:43 terraform-enterprise.log
-rw-r--r--. 1 terraform-enterprise root 3098 Oct 31 09:43 terraform-registry-api.log
-rw-r--r--. 1 terraform-enterprise root 2631 Oct 31 09:43 terraform-registry-worker.log
-rw-r--r--. 1 terraform-enterprise root 753 Oct 31 09:43 terraform-state-parser.log
-rw-r--r--. 1 terraform-enterprise root 116 Oct 31 09:42 tfe-health-check.log
-rw-r--r--. 1 terraform-enterprise root 8174 Oct 31 09:43 vault.log
Podman
- Alter the tfe.yaml file by adding this command
apiVersion: "v1"
kind: "Pod"
metadata:
labels:
app: "terraform-enterprise"
name: "terraform-enterprise"
spec:
restartPolicy: "Never"
containers:
- image: "images.releases.hashicorp.com/hashicorp/terraform-enterprise:v202410-1"
command: ["sleep", "infinity"]
- Stop and start the current container with the new settings
podman pod stop terraform-enterprise
podman pod rm terraform-enterprise
podman play kube /opt/tfe/tfe.yaml
- Connect to the container
podman exec -it terraform-enterprise-terraform-enterprise bash
- Start the process that will start the TFE application components
/usr/local/bin/supervisord-run
- If things crash you can review all the logs
terraform-enterprise@d4d1d593bcdc:/var/run$ ls -al /var/log/terraform-enterprise/
total 96
drwxrwxr-x. 2 root root 460 Oct 31 09:42 .
drwxr-xr-x. 1 root root 36 Sep 25 22:18 ..
-rw-r--r--. 1 terraform-enterprise root 1737 Oct 31 09:43 archivist.log
-rw-r--r--. 1 terraform-enterprise root 1673 Oct 31 09:43 atlas.log
-rw-r--r--. 1 terraform-enterprise root 1519 Oct 31 09:43 backup-restore.log
-rw-r--r--. 1 terraform-enterprise root 10036 Oct 31 09:43 licensing.log
-rw-r--r--. 1 terraform-enterprise root 774 Oct 31 09:43 metrics.log
-rw-r--r--. 1 nginx nginx 0 Oct 31 09:42 nginx-access.log
-rw-r--r--. 1 nginx nginx 0 Oct 31 09:42 nginx-error.log
-rw-r--r--. 1 terraform-enterprise root 0 Oct 31 09:42 nginx.log
-rw-r--r--. 1 terraform-enterprise root 215 Oct 31 09:43 outbound-http-proxy.log
-rw-r--r--. 1 terraform-enterprise root 2749 Oct 31 09:43 postgres.log
-rw-r--r--. 1 terraform-enterprise root 1616 Oct 31 09:43 redis.log
-rw-r--r--. 1 terraform-enterprise root 1978 Oct 31 09:43 sidekiq.log
-rw-r--r--. 1 terraform-enterprise root 966 Oct 31 09:43 slug-ingress.log
-rw-r--r--. 1 terraform-enterprise root 6133 Oct 31 09:43 supervisord.log
-rw-r--r--. 1 terraform-enterprise root 2125 Oct 31 09:43 task-worker.log
-rw-r--r--. 1 terraform-enterprise root 10808 Oct 31 09:43 terraform-enterprise.log
-rw-r--r--. 1 terraform-enterprise root 3098 Oct 31 09:43 terraform-registry-api.log
-rw-r--r--. 1 terraform-enterprise root 2631 Oct 31 09:43 terraform-registry-worker.log
-rw-r--r--. 1 terraform-enterprise root 753 Oct 31 09:43 terraform-state-parser.log
-rw-r--r--. 1 terraform-enterprise root 116 Oct 31 09:42 tfe-health-check.log
-rw-r--r--. 1 terraform-enterprise root 8174 Oct 31 09:43 vault.log
Kubernetes/Openshift
- The container would be in a crashing mode
kubectl -n terraform-enterprise get pods
NAME READY STATUS RESTARTS AGE
terraform-enterprise-69f4866769-h8lq4 0/1 CrashLoopBackOff 5 1h
- Alter the deployment
kubectl -n terraform-enterprise edit deployment terraform-enterprise
- Add the following command on container level
spec:
containers:
- image: images.releases.hashicorp.com/hashicorp/terraform-enterprise:v202410-1
command: ["sleep","infinity"]
- This should create a new pod starting that looks like the following. It doesn’t crash
kubectl -n terraform-enterprise get pods
NAME READY STATUS RESTARTS AGE
terraform-enterprise-69f4866769-hxlq5 0/1 Running 0 5m
- Connect to the pod
kubectl -n terraform-enterprise exec -it terraform-enterprise-69f4866769-h8lq4 -- bash
- Start the process that will start the TFE application components
/usr/local/bin/supervisord-run
- If things crash you can review all the logs
terraform-enterprise@d4d1d593bcdc:/var/run$ ls -al /var/log/terraform-enterprise/
total 96
drwxrwxr-x. 2 root root 460 Oct 31 09:42 .
drwxr-xr-x. 1 root root 36 Sep 25 22:18 ..
-rw-r--r--. 1 terraform-enterprise root 1737 Oct 31 09:43 archivist.log
-rw-r--r--. 1 terraform-enterprise root 1673 Oct 31 09:43 atlas.log
-rw-r--r--. 1 terraform-enterprise root 1519 Oct 31 09:43 backup-restore.log
-rw-r--r--. 1 terraform-enterprise root 10036 Oct 31 09:43 licensing.log
-rw-r--r--. 1 terraform-enterprise root 774 Oct 31 09:43 metrics.log
-rw-r--r--. 1 nginx nginx 0 Oct 31 09:42 nginx-access.log
-rw-r--r--. 1 nginx nginx 0 Oct 31 09:42 nginx-error.log
-rw-r--r--. 1 terraform-enterprise root 0 Oct 31 09:42 nginx.log
-rw-r--r--. 1 terraform-enterprise root 215 Oct 31 09:43 outbound-http-proxy.log
-rw-r--r--. 1 terraform-enterprise root 2749 Oct 31 09:43 postgres.log
-rw-r--r--. 1 terraform-enterprise root 1616 Oct 31 09:43 redis.log
-rw-r--r--. 1 terraform-enterprise root 1978 Oct 31 09:43 sidekiq.log
-rw-r--r--. 1 terraform-enterprise root 966 Oct 31 09:43 slug-ingress.log
-rw-r--r--. 1 terraform-enterprise root 6133 Oct 31 09:43 supervisord.log
-rw-r--r--. 1 terraform-enterprise root 2125 Oct 31 09:43 task-worker.log
-rw-r--r--. 1 terraform-enterprise root 10808 Oct 31 09:43 terraform-enterprise.log
-rw-r--r--. 1 terraform-enterprise root 3098 Oct 31 09:43 terraform-registry-api.log
-rw-r--r--. 1 terraform-enterprise root 2631 Oct 31 09:43 terraform-registry-worker.log
-rw-r--r--. 1 terraform-enterprise root 753 Oct 31 09:43 terraform-state-parser.log
-rw-r--r--. 1 terraform-enterprise root 116 Oct 31 09:42 tfe-health-check.log
-rw-r--r--. 1 terraform-enterprise root 8174 Oct 31 09:43 vault.log
Additional Information
- Terraform Enterprise FDO (Flexible Deployment Option) is documented here