Proxy with Basic Authentication blocking Terraform Enterprise runs on init

Problem

When a Terraform Enterprise instance is configured behind a proxy that requires Basic Authentication, terraform init commands fail with an authentication error.

│ Error: Failed to query available provider packages
│ 
│ Could not retrieve the list of available versions for provider
│ hashicorp/external: could not connect to registry.terraform.io: failed to
│ request discovery document: Get
│ "https://registry.terraform.io/.well-known/terraform.json":
│ AuthRequir

Direct requests to the registry from the same environment using curl may succeed, which can make diagnosis difficult.

$ curl "https://registry.terraform.io/.well-known/terraform.json"

Prerequisites

• A Terraform Enterprise environment is operating behind a proxy that uses Basic Authentication.
• Terraform Enterprise is configured to use the proxy. The following environment variables should be present in the container runtime configuration:

http_proxy=http://<user>:<pass>@<proxy_host>:8080
https_proxy=http://<user>:<pass>@<proxy_host>:8080
HTTP_PROXY=http://<user>:<pass>@<proxy_host>:8080
HTTPS_PROXY=http://<user>:<pass>@<proxy_host>:8080
no_proxy=localhost,127.0.0.1,<tfe_host_fqdn>

Cause

The proxy is blocking requests based on the User-Agent header sent by the Terraform binary. The Terraform binary, which is built with Go, uses the default Go-http-client user agent. Some proxies may be configured to block this user agent by default.

You can verify this behavior by running two diagnostic curl commands from a host behind the proxy.

1. Send a request with the default curl user agent. This command should succeed.

   $ curl -LI https://registry.terraform.io/.well-known/terraform.json -A "curl/8.1.2"

   The output shows a successful HTTP/2 200 response.

   HTTP/2 200
   content-type: application/json
   content-length: 62
   date: Thu, 26 Oct 2023 15:40:02 GMT
   ##...

2. Send a request that mimics the Terraform binary by setting the user agent to Go-http-client.

   $ curl -LI https://registry.terraform.io/.well-known/terraform.json -A "Go-http-client/1.1"

   This command fails with an HTTP/1.1 407 AuthRequir error, confirming the proxy is blocking this specific user agent.

   HTTP/1.1 407 AuthRequir
   content-type: application/json
   content-length: 1
   date: Thu, 26 Oct 2023 15:40:02 GMT
   cache-control: no-cache
   x-frame-options: deny
   proxy-connection: keep-alive
   proxy-authenticate: negotiate

Solutions

Solution 1: Allow the 'Go-http-client' User Agent

Consult your networking or security team to update the proxy configuration. The proxy must be configured to allow requests that use the Go-http-client user agent.

Outcome

After the proxy is configured to allow the Go-http-client user agent, terraform init commands will complete successfully.

Additional Information

• For more details on configuring Terraform Enterprise with a proxy, refer to the Proxy Usage documentation.