Troubleshooting Terraform Enterprise Application Not Running After System Update – HashiCorp Help Center

Introduction

After performing a system or kernel patch, the Terraform Enterprise (TFE) application deployed with Replicated may fail to start. This can stem from configuration changes, service disruptions, or environmental conflicts.

Prerequisites

Access to the TFE host
Permissions to execute system commands and modify configurations
Understanding of Docker and systemd services

Common Causes

Disabled IPv4 forwarding
Docker not running or misconfigured
Incorrect Replicated user permissions
Firewall rules blocking Docker or Replicated traffic
Manual edits to service files causing version mismatch
Conflicting software or dependencies after updates

Troubleshooting Steps

1. Check IPv4 Forwarding

sysctl net.ipv4.ip_forward

If the output is 0, enable it:

Edit /etc/sysctl.conf and add:

net.ipv4.ip_forward = 1
Apply changes:

sysctl -p systemctl restart network

2. Verify Docker Status

docker info
sudo systemctl daemon-reload
sudo service docker restart
systemctl status docker
docker system df
df -h
docker ps

3. Collect Logs for Analysis

journalctl --xu docker --since "YYYY-MMM-DD hh:mm:ss" > docker_<date>.log
journalctl -u replicated.service --since "YYYY-MMM-DD hh:mm:ss" > replicated_<date>.log

4. Inspect Firewall Rules

List current rules:

sudo iptables -L -v -n

Host-level rules

Block outbound networking:

sudo iptables -A OUTPUT -p tcp --dport 443 -j DROP
sudo iptables -A OUTPUT -p tcp --dport 80 -j DROP

To revert (remove) the block:

sudo iptables -D OUTPUT -p tcp --dport 443 -j DROP
sudo iptables -D OUTPUT -p tcp --dport 80 -j DROP

List DROP rules (host level):

sudo iptables -L -v -n --line-numbers | grep DROP

Remove DROP rules by rule number (example removing first two):

sudo iptables -D OUTPUT 1
sudo iptables -D OUTPUT 1

Container-level rules (DOCKER-USER chain)

To Block outbound (example):

sudo iptables -I DOCKER-USER -p tcp --dport 443 -j DROP
sudo iptables -I DOCKER-USER -p tcp --dport 80 -j DROP

List rules:

sudo iptables -L DOCKER-USER --line-numbers

Remove a rule (example for rule #1):

sudo iptables -D DOCKER-USER 1

5. Inspect System Logs

tail -n 100 /var/log/syslog | grep "error"
tail -n 100 /var/log/messages | grep "error"

6. Check Resource Availability

free -h
df -h
top

7. Validate Network Configuration

ip addr show

8. Check User and Group Permissions

Verify user:

cat /etc/passwd | grep replicated
Check Docker socket permissions:

ls -la /var/run/docker.sock
Add replicated user to Docker group if missing:

sudo usermod -aG docker replicated groups replicated