Problem
When running a Terraform Enterprise Active-Active installation in a locked-down environment with no inbound connectivity, build workers may fail to initialize. If the load balancer does not automatically route outbound requests from internal services back to other internal addresses (a technique known as hairpinning), runs will fail.
The following error message appears in the run logs, indicating the build worker cannot reach the tfe-atlas container (or ptfe_atlas in versions v202204-1 and earlier) using the installation's external hostname.
Error: Failed to request discovery document: Get "https://<TFE_HOSTNAME>/.well-known/terraform.json": context deadline exceeded on zzz_backend_override.tf.json line 5, in terraform[0].backend[0].remote: 5: "hostname": "<TFE_HOSTNAME>",
Cause
Terraform Enterprise build workers connect to the tfe-atlas container using the installation's fully qualified domain name (FQDN) by default. In environments without hairpinning enabled at the network level, the worker's request to the external FQDN cannot resolve back to the internal IP address of the tfe-atlas container, causing the connection to time out.
Solutions
This issue can be resolved by enabling hairpinning within the Docker network, using a proxy, or by overriding DNS at the container level.
Solution 1: Enable Hairpin Addressing
This is the recommended solution. You can configure Terraform Enterprise to handle this network routing internally.
-
Set the
hairpin_addressingvalue to1using thetfe-admincommand on the primary node. For automated installations, you can set the hairpin_addressing configuration key.$ tfe-admin app-config -k hairpin_addressing -v "1"
-
Stop the Terraform Enterprise application on all nodes.
$ replicatedctl app stop
-
Monitor the application status to confirm it has fully stopped.
$ watch replicatedctl app status
-
Start the Terraform Enterprise application on all nodes.
$ replicatedctl app start
Solution 2: Use a Proxy Sandwich
Configure a proxy to intercept the outbound request from the build worker and route it back to the internal load balancer for the Terraform Enterprise application. This effectively performs the hairpinning function at the proxy layer instead of within the application's network.
Solution 3: Use a DNS Override
Use a tool like dnsmasq to override DNS resolution at the container level. This forces requests to the external FQDN to resolve to the internal IP address of the application, ensuring the build worker can connect to the tfe-atlas container directly.