Terraform Enterprise FDO proxy settings

Problem

In certain environments, the Terraform Enterprise host lacks direct internet access. A proxy is required for operations such as the terraform init stage of a run, which downloads necessary providers and binaries.

Without a correctly configured proxy, you may encounter an error message similar to the following when Terraform Enterprise cannot connect to the internet.

Operation failed: failed fetching Terraform: failed downloading terraform: failed downloading "https://releases.hashicorp.com/terraform/1.5.3/terraform_1.5.3_linux_amd64.zip": GET https://releases.hashicorp.com/terraform/1.5.3/terraform_1.5.3_linux_amd64.zip giving up after 5 attempt(s): failed making temp file: open /tmp/terraform/71bf0bd0bd10f6eeac2261e23543df18.download-504ba21b-3a45-c1d4-ff08-7cd03823c4d8: read-only file system

Solutions

This article provides proxy configuration instructions for different Terraform Enterprise Flexible Deployment Options (FDO).

Solution 1: Docker Deployment

Add proxy environment variables to the environment section of your docker-compose.yml file, as described in the Docker deployment documentation.

---
version: "3.9"
name: terraform-enterprise
services:
  tfe:
    image: images.releases.hashicorp.com/hashicorp/terraform-enterprise:v202309-1
    environment:
      TFE_HOSTNAME: tfe.domain.com
      http_proxy: http://<your_proxy_server>:8080
      https_proxy: http://<your_proxy_server>:8080
      no_proxy: localhost,127.0.0.1,169.254.169.24,your_tfe_fqdn>

Restart Terraform Enterprise to apply the new settings.

$ sudo docker compose -f docker_compose_file.yml restart

Solution 2: Kubernetes Deployment

Add proxy environment variables to the overrides.yaml file of the Terraform Enterprise Helm chart.

env:
  variables:
    https_proxy: http://<your_proxy_server>:8080
    http_proxy: http://<your_proxy_server>:8080
    no_proxy: localhost,127.0.0.1,169.254.169.254,<your_tfe_fqdn>,<KUBERNETES_SERVICE_HOST>

Finding the KUBERNETES_SERVICE_HOST

You must add the KUBERNETES_SERVICE_HOST IP address to the no_proxy list to allow internal cluster communication. For more details, refer to the Kubernetes documentation on accessing the API from a Pod.

To find the IP address, connect to a running pod on the cluster and check the environment variable.

$ env | grep KUBERNETES_SERVICE_HOST
## KUBERNETES_SERVICE_HOST=172.20.0.1

If you do not add the KUBERNETES_SERVICE_HOST to no_proxy, you may see the following error in the logs when starting a run.

{
  "@level": "error",
  "@message": "error running task instance",
  "@module": "task-worker.executor",
  "err": "error creating kubernetes job: Post \"https://172.20.0.1:443/apis/batch/v1/namespaces/terraform-enterprise-agents/jobs\": context deadline exceeded"
}

Solution 3: Podman Deployment

Add proxy environment variables to the env section of your Podman kube.yml file, as described in the Podman deployment documentation.

- env:
    ## Proxy settings
    - name: "https_proxy"
      value: "http://<your_proxy_server>:8080"
    - name: "http_proxy"
      value: "http://<your_proxy_server>:8080"
    - name: "no_proxy"
      value: "localhost,127.0.0.1,169.254.169.254,<your_tfe_fqdn>"

Additional Information

Agent Proxy Inheritance

Agents that start as part of a workspace run automatically inherit the proxy settings from the main Terraform Enterprise configuration. This is also true when using a custom agent image. The following example shows how proxy variables are passed alongside a custom agent image definition.

env:
  variables:
    https_proxy: http://<your_proxy_server>:8080
    http_proxy: http://<your_proxy_server>:8080
    no_proxy: localhost,127.0.0.1,169.254.169.254,<your_tfe_fqdn>,<KUBERNETES_SERVICE_HOST>
    TFE_RUN_PIPELINE_IMAGE: example/custom-agent:latest