Introduction
Problem
When your TFE is behind a proxy where Basic Authentication is required, 'terraform init' fails with the following error:
│ Error: Failed to query available provider packages
│
│ Could not retrieve the list of available versions for provider
│ hashicorp/external: could not connect to registry.terraform.io: failed to
│ request discovery document: Get
│ "https://registry.terraform.io/.well-known/terraform.json": AuthRequir
While curl is working fine:
curl "https://registry.terraform.io/.well-known/terraform.json"
Prerequisites
- A TFE environment behind a proxy with Basic Authentication
- TFE is configured to use the proxy
The following environment variable should be present:
http_proxy=http://<user>:<pass>@<proxy_host>:8080
https_proxy=http://<user>:<pass>@<proxy_host>:8080
HTTP_PROXY=http://<user>:<pass>@<proxy_host>:8080
HTTPS_PROXY=http://<user>:<pass>@<proxy_host>:8080
no_proxy=localhost,127.0.0.1,<tfe_host_fqdn>
Cause
-
The proxy is blocking requests based on User Agent.
From a host behind the proxy, run the following two curl commands:
1) Curl pretending to be User Agent 'Curl':
curl -LI https://registry.terraform.io/.well-known/terraform.json -A "curl/8.1.2"
Sample output:
HTTP/2 200 content-type: application/json content-length: 62 date: Thu, 26 Oct 2023 15:40:02 GMT accept-ranges: bytes cache-control: max-age=3600, stale-if-error=31536000, public content-security-policy: default-src 'self'; script-src 'self' 'unsafe-inline' https://www.google-analytics.com https://cdn.segment.com https://unpkg.com/@segment/consent-manager@5.6.0/standalone/consent-manager.js https://www.googletagmanager.com https://a.optnmstr.com; style-src 'self' 'unsafe-inline' https://maxcdn.bootstrapcdn.com https://fonts.googleapis.com https://p.typekit.net https://use.typekit.net; img-src 'self' data: https: https://www.google-analytics.com; font-src 'self' https://maxcdn.bootstrapcdn.com https://fonts.googleapis.com https://fonts.gstatic.com https://use.typekit.net; connect-src 'self' https://www.google-analytics.com https://*.launchdarkly.com https://api.segment.io https://cdn.segment.com https://sentry.io https://api.omappapi.com https://api.opmnstr.com https://api.optmnstr.com https://*.algolia.net https://*.algolianet.com https://app.terraform.io https://app.staging.terraform.io https://api.github.com/emojis feature-policy: last-modified: Thu, 26 Oct 2023 07:26:07 GMT referrer-policy: no-referrer-when-downgrade server: terraform-registry/736a990c1b1e3c7bc370d93a7e059c2a8fe573fb strict-transport-security: max-age=31536000; includeSubDomains; preload x-content-type-options: nosniff x-frame-options: DENY x-xss-protection: 1; mode=block vary: Accept-Encoding x-cache: Hit from cloudfront via: 1.1 3581c46f65a0770e1977ff05d5b7ad80.cloudfront.net (CloudFront) x-amz-cf-pop: DUB56-P1 x-amz-cf-id: _s5YfBNqm0_eHmgSVrm5HJXeMdd0oOOWTNroMwbWkSp4FRdn6JC0uQ== age: 1325
This curl command succeeds.
2) Curl pretending to be User Agent 'Go-http-client'. With this we mimic the terraform binary
curl -LI https://registry.terraform.io/.well-known/terraform.json -A "Go-http-client/1.1"
Sample output:
HTTP/1.1 407 AuthRequir content-type: application/json content-length: 1 date: Thu, 26 Oct 2023 15:40:02 GMT cache-control: no-cache x-frame-options: deny
proxy-connection: keep-alive
proxy-authenticate: negotiate
This curl command fails with a 'HTTP error 407 AuthRequir'.
Running this curl commands from a non-proxied host succeeds.
Overview of possible solutions
Solutions:
-
Consult your networking team to get the User Agent 'Go-http-client' unblocked.
Outcome
When the User Agent is unblocked, 'terraform init' runs should normally run again.