Introduction
Workspaces in Terraform Enterprise are configured to execute Terraform runs. During a run, Terraform CLI will communicate with the API of Terraform Enterprise itself as described in the discovery process documentation. Terraform Enterprise executes these Terraform runs in an ephemeral container either using the default worker image or a custom worker image.
Problem
A Terraform run failed with the following error:
Error: Service discovery failed for terraform.example.com
Failed to request discovery document: Get
https://terraform.example.com/.well-known/terraform.json: x509: certificate signed by unknown authority.
Cause
Terraform Enterprise may be configured with TLS certificates that are not publicly-trusted or distributed with the Docker image that is used to execute the Terraform run.
Solution
There are two solutions to this problem. One when using the default worker image and the other when using a custom worker image.
Default Worker Image
When using the default worker image, it is required to add all necessary TLS certificates into the CA certificate bundle used by Terraform Enterprise. The settings for the CA certificate bundle are located on the Replicated console on port 8800 of your installation, within the “Settings” tab, and under the “SSL/TLS Configuration” section. When adding TLS certificates to the CA certificate bundle, the TLS certificates must be PEM-encoded. An example of what a PEM-encoded TLS certificate looks like is below.
-----BEGIN CERTIFICATE-----
MIIGUTCCBDmgAwIBAgIUTwHgVBQzf8hE77Lnd6H+7Luuqv4wDQYJKoZIhvcNAQEL
BQAwgbcxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQH
...
7gAlP1gpguPHUkPaylJ+e74iQPMMUBzW1ELIdbZ4dooCwAdSnEoeUE0Rz+AHWJZ7
ch+Oc0AT/bVCiHjFsoBh7Y+TE9aWfK7A90oUQhvPDMdiKPcKtQ==
-----END CERTIFICATE-----
More information regarding updating the CA certificate bundle can be found here.
Once the CA certificate bundle settings are changed, please scroll to the bottom of the page and click the “Save” button. If prompted, restart the Terraform Enterprise application. Otherwise, follow the instructions below to manually restart the Terraform Enterprise application.
From the Replicated console on port 8800, navigate to the “Dashboard” tab and click on the “Stop Now” button. Once the application has stopped, click on the “Start Now” button to restart it.
Custom Worker Image
With custom worker image, the certificates in the chain are required to be present at /etc/ssl/certs/ca-certificates.crt.
More information regarding custom worker image can be found here.
Additional Information
Self Signed Certificates
Self-signed certificates can prevent Terraform Enterprise from connecting to the remote backend. This can be resolved by importing the CA certificates that were used to sign the self-signed certificate into the local machine.
MacOS
Users running macOS Catalina reported receiving the x509: certificate signed by unknown authorityerror even though the TLS certificate chain was valid. There was a change in macOS that prevented Go binaries from reading certificates held inside Keychain. The fix for this is documented here.